Context-aware AR instruction enables adaptive and in-situ learning experiences. However, hardware limitations and expertise requirements constrain the creation of such instructions. With recent developments in Generative Artificial Intelligence (Gen-AI), current research tries to tackle these constraints by deploying AI-generated content (AIGC) in AR applications. However, our preliminary study with six AR practitioners revealed that the current AIGC lacks
contextual information to adapt to varying application scenarios and is therefore limited in authoring. To utilize the strong generative power of GenAI to ease the authoring of AR instruction while capturing the context, we developed CARING-AI, an AR system to author context-aware humanoid-avatar-based instructions with GenAI. By navigating in the environment, users naturally provide contextual information to generate humanoid-avatar animation
as AR instructions that blend in the context spatially and temporally. We showcased three application scenarios of CARING-AI: Asynchronous Instructions, Remote Instructions, and Ad Hoc Instructions based on a design space of AIGC in AR Instructions.
CARING-AI: Towards Authoring Context-aware Augmented Reality INstruction through Generative Artificial Intelligence
Authors: Jingyu Shi*, Rahul Jain*, Seung-gun Chi*, Hyungjun Doh, Hyung-gun Chi, Alexander J. Quinn and Karthik Ramani
CHI Conference on Human Factors in Computing Systems (CHI ’25)
https://doi.org/10.1145/3706598.3713348
Rahul Jain
Rahul Jain has been a Ph.D. student in the School of Electrical and Computer Engineering at Purdue University since Spring 2022. He is conducting research under Professor Karthik Ramani’s Convergence Design Lab. He received his Master’s in Electrical and Computer Engineering at Purdue University and Bachelor’s in Civil Engineering at Indian Institute of Technology (IIT), Patna. His current research focuses on area of Computer Vision, Machine Learning and human-computer interactions utilizing AR/VR.