Ziyi Liu

Ziyi Liu

Ziyi Liu has been a Ph.D. student in the School of Mechanical Engineering at Purdue University since Fall 2021. He is conducting research under Professor Karthik Ramani's Convergence Design Lab. He received his Master's and Bachelor's degrees in Mechanical Engineering at Purdue University. His current research focuses on innovative human-computer interactions utilizing AR/VR, and AI in authoring/tutoring systems.
Towards Design Guidelines for Natural-Language-Based Human-Multi-Robot Collaboration in Domestic Environments

Towards Design Guidelines for Natural-Language-Based Human-Multi-Robot Collaboration in Domestic Environments

Xinyi Wang, Shao-Kang Hsia, Ziyi Liu, Chenfei Zhu, Zhengzhe Zhu, Xiyun Hu, Anastasia Kouvaras Ostrowski, Karthik Ramani
In Proceedings of the Extended Abstracts of the 2026 CHI Conference on Human Factors in Computing Systems

As domestic robots become prevalent, households will involve multiple robots with different capabilities, requiring people to coordinate and collaborate with several robots daily. This demands understanding key human–multi-robot interaction that...

JustShape: Exploring Co-Speech Gestures for Multimodal LLM-Powered 3D Parametric Modeling

JustShape: Exploring Co-Speech Gestures for Multimodal LLM-Powered 3D Parametric Modeling

Runlin Duan, Yuzhao Chen, Yichen Hu, Ziyi Liu, Chenfei Zhu, Xiyun Hu, Dizhi Ma, Xinyi Wang, Karthik Ramani
In Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems

Parametric modeling is a prevailing 3D modeling approach in design, architecture, and engineering. The emergence of multimodal large language models (LLMs) brings a new opportunity to lower the entry barriers to this powerful tool. However,...

AmIWrite: Exploring Scalable One-on-One Handwriting-Based Tutoring for Mathematical Problem-Solving with an LLM-Powered AI Tutor

AmIWrite: Exploring Scalable One-on-One Handwriting-Based Tutoring for Mathematical Problem-Solving with an LLM-Powered AI Tutor

Ziyi Liu, Yuzhao Chen, Haoyu Ji, Runlin Duan, Zhengzhe Zhu, Xiyun Hu, Kylie Peppler, Karthik Ramani
In Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems

Real-time handwriting interactions between tutors and students —where tutors observe individual problem-solving processes, provide personalized annotations, and adapt explanations based on students’ work—are fundamental to effective STEM tutoring....

agentAR: Creating Augmented Reality Applications with Tool-Augmented LLM-based Autonomous Agents

agentAR: Creating Augmented Reality Applications with Tool-Augmented LLM-based Autonomous Agents

Chenfei Zhu, Shao-Kang Hsia, Xiyun Hu, Ziyi Liu, Jingyu Shi, Karthik Ramani
In Proceedings of the 38th Annual ACM Symposium on User Interface Software and Technology (UIST '25).

Creating Augmented Reality (AR) applications requires expertise in both design and implementation, posing significant barriers to entry for non-expert users. While existing methods reduce some of this burden, they often fall short in flexibility or...

GesPrompt: Leveraging Co-Speech Gestures to Augment LLM-Based Interaction in Virtual Reality

GesPrompt: Leveraging Co-Speech Gestures to Augment LLM-Based Interaction in Virtual Reality

Xiyun Hu, Dizhi Ma, Fengming He, Zhengzhe Zhu, Shao-Kang Hsia, Chenfei Zhu, Ziyi Liu, Karthik Ramani
In Proceedings of the 2025 ACM Designing Interactive Systems Conference

Large Language Model (LLM)-based copilots have shown great potential in Extended Reality (XR) applications. However, the user faces challenges when describing the 3D environments to the copilots due to the complexity of conveying spatial-temporal...

Virtual Reality in Manufacturing Education: A Scoping Review Indicating State-of-the-Art, Benefits, and Challenges Across Domains, Levels, and Entities

Virtual Reality in Manufacturing Education: A Scoping Review Indicating State-of-the-Art, Benefits, and Challenges Across Domains, Levels, and Entities

Ananya Ipsita, Ramesh Kaki, Ziyi Liu, Mayank Patel, Runlin Duan, Lakshmi Deshpande, Lin-Ping Yuan, Victoria Lowell, Ashok Maharaj, Kylie Peppler, Steven Feiner, Karthik Ramani
Preprint submitted to Elsevier

To address the shortage of a skilled workforce in the U.S. manufacturing industry, immersive Virtual Reality (VR)-based training solutions hold promising potential. To effectively utilize VR to meet workforce demands, it is important to understand...

ClassMeta: Designing Interactive Virtual Classmate to Promote VR Classroom Participation

ClassMeta: Designing Interactive Virtual Classmate to Promote VR Classroom Participation

Ziyi Liu*, Zhengzhe Zhu*, Lijun Zhu, Enze Jiang, Xiyun Hu, Kylie A Peppler, Karthik Ramani
In Proceedings of the CHI Conference on Human Factors in Computing Systems, pp. 1-17. 2024.

Peer influence plays a crucial role in promoting classroom participation, where behaviors from active students can contribute to a collective classroom learning experience. However, the presence of these active students depends on several...

InstruMentAR: Auto-Generation of Augmented Reality Tutorials for Operating Digital Instruments Through Recording Embodied

InstruMentAR: Auto-Generation of Augmented Reality Tutorials for Operating Digital Instruments Through Recording Embodied

Ziyi Liu, Zhengzhe Zhu, Enze Jiang, Feichi Huang, Ana Villanueva, Tianyi Wang, Xun Qian, Karthik Ramani
In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems

Augmented Reality tutorials, which provide necessary context by directly superimposing visual guidance on the physical referent, represent an effective way of scaffolding complex instrument operations. However, current AR tutorial authoring...

LearnIoTVR: An End-to-End Virtual Reality Environment Providing Authentic Learning Experiences for Internet of Things

LearnIoTVR: An End-to-End Virtual Reality Environment Providing Authentic Learning Experiences for Internet of Things

Zhengzhe Zhu, Ziyi Liu, Youyou Zhang, Lijun Zhu, Joey Huang, Ana villanueva, Xun Qian, Kylie Peppler, Karthik Ramani
In the Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems

The rapid growth of Internet-of-Things (IoT) applications has generated interest from many industries and a need for graduates with relevant knowledge. An IoT system is comprised of spatially distributed interactions between humans and various...