Towards Design Guidelines for Natural-Language-Based Human-Multi-Robot Collaboration in Domestic Environments

Towards Design Guidelines for Natural-Language-Based Human-Multi-Robot Collaboration in Domestic Environments

Xinyi Wang, Shao-Kang Hsia, Ziyi Liu, Chenfei Zhu, Zhengzhe Zhu, Xiyun Hu, Anastasia Kouvaras Ostrowski, Karthik Ramani
In Proceedings of the Extended Abstracts of the 2026 CHI Conference on Human Factors in Computing Systems

As domestic robots become prevalent, households will involve multiple robots with different capabilities, requiring people to coordinate and collaborate with several robots daily. This demands understanding key human–multi-robot interaction that...

ARify: Leveraging Narrated Instructional Videos to Create Augmented Reality Tutorials for Procedural Tasks

ARify: Leveraging Narrated Instructional Videos to Create Augmented Reality Tutorials for Procedural Tasks

Xiyun Hu, Chenfei Zhu, Shao-Kang Hsia, Dizhi Ma, Rahul Jain, Karthik Ramani
In Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems

Augmented Reality (AR) tutorials enhance procedural task learning by providing situated, step-by-step guidance. Yet, creating such tutorials requires AR authoring expertise, posing a significant entry barrier. To lower this barrier, we introduce...

JustShape: Exploring Co-Speech Gestures for Multimodal LLM-Powered 3D Parametric Modeling

JustShape: Exploring Co-Speech Gestures for Multimodal LLM-Powered 3D Parametric Modeling

Runlin Duan, Yuzhao Chen, Yichen Hu, Ziyi Liu, Chenfei Zhu, Xiyun Hu, Dizhi Ma, Xinyi Wang, Karthik Ramani
In Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems

Parametric modeling is a prevailing 3D modeling approach in design, architecture, and engineering. The emergence of multimodal large language models (LLMs) brings a new opportunity to lower the entry barriers to this powerful tool. However,...

AmIWrite: Exploring Scalable One-on-One Handwriting-Based Tutoring for Mathematical Problem-Solving with an LLM-Powered AI Tutor

AmIWrite: Exploring Scalable One-on-One Handwriting-Based Tutoring for Mathematical Problem-Solving with an LLM-Powered AI Tutor

Ziyi Liu, Yuzhao Chen, Haoyu Ji, Runlin Duan, Zhengzhe Zhu, Xiyun Hu, Kylie Peppler, Karthik Ramani
In Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems

Real-time handwriting interactions between tutors and students —where tutors observe individual problem-solving processes, provide personalized annotations, and adapt explanations based on students’ work—are fundamental to effective STEM tutoring....

AgentCoach: LLM-Based Adaptive Coaching Feedback for Motor Skill Learning

AgentCoach: LLM-Based Adaptive Coaching Feedback for Motor Skill Learning

Dizhi Ma, Jiakun Yu, Xinyi Wang, Xiyun Hu, Liang He, Sooyeon Jeong, Karthik Ramani
In Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems

We present AgentCoach, an LLM-powered system that provides adaptive feedback for motor skill learning from tutorial videos. The system works by extracting key coaching points (CPs) and compiling CP-specific evaluators that map each cue to...

Canvas3D: Empowering Precise Spatial Control for Image Generation with Constraints from a 3D Virtual Canvas

Canvas3D: Empowering Precise Spatial Control for Image Generation with Constraints from a 3D Virtual Canvas

Yuzhao Chen, Runlin Duan, Rahul Jain, Yichen Hu, Chenfei Zhu, Jingyu Shi, Karthik Ramani
In Proceedings of the 31st International Conference on Intelligent User Interfaces

Generative AI (GenAI) has significantly advanced the ease and flexibility of image creation. However, it remains a challenge to precisely control spatial compositions, including object arrangement and scene conditions. To bridge this gap, we...

PACE: Physics Augmentation for Coordinated End-to-end Reinforcement Learning toward Versatile Humanoid Table Tennis

PACE: Physics Augmentation for Coordinated End-to-end Reinforcement Learning toward Versatile Humanoid Table Tennis

Muqun Hu, Wenxi Chen, Wenjing Li, Falak Mandali, Zijian He, Renhong Zhang, Praveen Krisna, Katherine Christian, Leo Benaharon, Dizhi Ma, Karthik Ramani, Yan Gu
2026 IEEE International Conference on Robotics and Automation (ICRA)

Humanoid table tennis (TT) demands rapid perception, proactive whole-body motion, and agile footwork under strict timing—capabilities that remain difficult for end-to-end control policies. We propose a reinforcement learning (RL) framework that...

ConceptVis: Exploring and Visualizing Design Concepts With Large Language Models Using Interactive Knowledge Graph

ConceptVis: Exploring and Visualizing Design Concepts With Large Language Models Using Interactive Knowledge Graph

Runlin Duan, Nachiketh Karthik, Yuzhao Chen, Jingyu Shi, Rahul Jain, Maria Yang, Karthik Ramani
ASME. J. Comput. Inf. Sci. Eng. January 2026; 26(1): 011002.

Large language models (LLMs) are capable of generating cross-domain design knowledge, opening up new possibilities for creating a myriad of design concepts for early-stage design ideation. However, the current chat-based interface fails to...