Towards Design Guidelines for Natural-Language-Based Human-Multi-Robot Collaboration in Domestic Environments

by Convergence Design Lab Admin | Apr 13, 2026 | 2026, Chenfei Zhu, Shao-Kang Hsia, Xinyi Wang, Xiyun Hu, Zhengzhe Zhu, Ziyi Liu

As domestic robots become prevalent, households will involve multiple robots with different capabilities, requiring people to coordinate and collaborate with several robots daily. This demands understanding key human–multi-robot interaction that shapes human...

JustShape: Exploring Co-Speech Gestures for Multimodal LLM-Powered 3D Parametric Modeling

by Runlin Duan | Apr 13, 2026 | 2026, Chenfei Zhu, Dizhi Ma, Karthik Ramani, Runlin Duan, Xiyun Hu, Yuzhao Chen, Ziyi Liu

Parametric modeling is a prevailing 3D modeling approach in design, architecture, and engineering. The emergence of multimodal large language models (LLMs) brings a new opportunity to lower the entry barriers to this powerful tool. However, describing 3D geometries...

AmIWrite: Exploring Scalable One-on-One Handwriting-Based Tutoring for Mathematical Problem-Solving with an LLM-Powered AI Tutor

by Ziyi Liu | Apr 13, 2026 | 2026, Karthik Ramani, Runlin Duan, Xiyun Hu, Yuzhao Chen, Zhengzhe Zhu, Ziyi Liu

Real-time handwriting interactions between tutors and students —where tutors observe individual problem-solving processes, provide personalized annotations, and adapt explanations based on students’ work—are fundamental to effective STEM tutoring. However, scaling...

agentAR: Creating Augmented Reality Applications with Tool-Augmented LLM-based Autonomous Agents

by Chenfei Zhu | Sep 30, 2025 | 2025, Chenfei Zhu, Featured Publications, Jingyu Shi, Karthik Ramani, Recent Publications, Shao-Kang Hsia, Xiyun Hu, Ziyi Liu

Creating Augmented Reality (AR) applications requires expertise in both design and implementation, posing significant barriers to entry for non-expert users. While existing methods reduce some of this burden, they often fall short in flexibility or usability for...

GesPrompt: Leveraging Co-Speech Gestures to Augment LLM-Based Interaction in Virtual Reality

by Xiyun Hu | May 14, 2025 | 2025, Dizhi Ma, Fengming He, Karthik Ramani, Karthik Ramani, Recent Publications, Shao-Kang Hsia, Xiyun Hu, Zhengzhe Zhu, Ziyi Liu

Large Language Model (LLM)-based copilots have shown great potential in Extended Reality (XR) applications. However, the user faces challenges when describing the 3D environments to the copilots due to the complexity of conveying spatial-temporal information through...

Virtual Reality in Manufacturing Education: A Scoping Review Indicating State-of-the-Art, Benefits, and Challenges Across Domains, Levels, and Entities

by Ananya Ipsita | Mar 25, 2025 | 2025, Ananya Ipsita, Karthik Ramani, Runlin Duan, Ziyi Liu

To address the shortage of a skilled workforce in the U.S. manufacturing industry, immersive Virtual Reality (VR)-based training solutions hold promising potential. To effectively utilize VR to meet workforce demands, it is important to understand the role of VR in...