Ubi Edge: Authoring Edge-Based Opportunistic Tangible User Interfaces in Augmented Reality

by | Feb 24, 2023

Authors: Fengming He, Xiyun Hu, Jingyu Shi, Xun Qian, Tianyi Wang, Karthik Ramani
In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems

Edges are one of the most ubiquitous geometric features of physical objects. They provide accurate haptic feedback and easy-totrack features for camera systems, making them an ideal basis for Tangible User Interfaces (TUI) in Augmented Reality (AR). We introduce Ubi Edge, an AR authoring tool that allows end-users to customize edges on daily objects as TUI inputs to control varied digital functions. We develop an integrated AR-device and an integrated vision-based detection pipeline that can track 3D edges and detect the touch interaction between fingers and edges. Leveraging the spatial-awareness of AR, users can simply select an edge by sliding fingers along it and then make the edge interactive by connecting it to various digital functions. We demonstrate four use cases including multi-function controllers, smart homes, games, and TUI-based tutorials. We also evaluated and proved our system’s usability through a two-session user study, where qualitative and quantitative results are positive.



Fengming He

Fengming He

Fengming He is a Ph.D. student in the School of Electrical and Computer Engineering at Purdue University. She received her B.S. degree in Electrical and Computer Engineering in 2017 from Wuhan University, China. Her research interest includes computer vision, machine learning and its application to Augmented Reality (AR) and Human-Computer Interaction (HCI).