With the advents in geometry perception and Augmented Reality (AR), end-users can customize Tangible User Interfaces (TUIs) that control digital assets using intuitive and comfortable interactions with physical geometries (e.g., edges and surfaces). However, it remains challenging to adapt such TUIs in varied physical environments while maintaining the same spatial and ergonomic affordance. We propose AdapTUI, an end-to- end system that enables an end-user to author geometric-based TUIs and automatically adapts the TUIs when the user moves to a new environment. Leveraging a geometry detection module and the spatial awareness of AR, AdapTUI first lets users create custom mappings between geometric features and digital functions. Then, AdapTUI uses an optimization-based adaptation framework, which considers both the geometric variations and human-factor nuances, to dynamically adjust the attachment of the user-authored TUIs. We demonstrate three application scenarios where end-users can utilize TUIs at different locations, including portable car play, efficient AR workstation, and entertainment. We evaluated the effectiveness of the adaptation method as well as the overall usability through a comparison user study (N=12). The satisfactory adaptation of the user-authored TUIs and the positive qualitative feedback demonstrate the effectiveness of our system.
AdapTUI: Adaptation of Geometric-Feature-Based Tangible User Interfaces in Augmented Reality
Authors: Fengming He, Xiyun Hu, Xun Qian, Zhengzhe Zhu, Karthik Ramani
In Proceedings of the ACM on Human-Computer Interaction, 8(ISS), 44-69.
https://doi.org/10.1145/3698127


Fengming He
Fengming He is a Ph.D. student in the School of Electrical and Computer Engineering at Purdue University. She received her B.S. degree in Electrical and Computer Engineering in 2017 from Wuhan University, China. Her research interest includes computer vision, machine learning and its application to Augmented Reality (AR) and Human-Computer Interaction (HCI).