MechARspace: An Authoring System Enabling Bidirectional Binding of Augmented Reality with Toys in Real-time

by | Nov 10, 2022

Authors: Zhengzhe Zhu, Ziyi Liu, Tianyi Wang, Youyou Zhang, Xun Qian, Pashin Farsak Raja, Ana Villanueva, Karthik Ramani
In Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology (pp. 1-16).

Augmented Reality (AR), which blends physical and virtual worlds, presents the possibility of enhancing traditional toy design. By leveraging bidirectional virtual-physical interactions between humans and the designed artifact, such AR-enhanced toys can provide more playful and interactive experiences for traditional toys. However, designers are constrained by the complexity and technical difficulties of the current AR content creation processes. We propose MechARspace, an immersive authoring system that supports users to create toy-AR interactions through direct manipulation and visual programming. Based on the elicitation study, we propose a bidirectional interaction model which maps both ways: from the toy inputs to reactions of AR content, and also from the AR content to the toy reactions. This model guides the design of our system which includes a plug-and-play hardware toolkit and an in-situ authoring interface. We present multiple use cases enabled by MechARspace to validate this interaction model. Finally, we evaluate our system with a two-session user study where users first recreated a set of predefined toy-AR interactions and then implemented their own AR-enhanced toy designs.

Zhengzhe Zhu

Zhengzhe Zhu

Zhengzhe Zhu is a first year PhD student in the school of ECE at Purdue University. His current work focuses on Cloud-based Human-Computer Interaction (HCI). Currently, he is working on an online multi-client collaboration platform for the intelligent tutoring system.