Interacting Objects: A dataset of object-object interactions for richer dynamic scene representations

Interacting Objects: A dataset of object-object interactions for richer dynamic scene representations

Asim Unmesh, Rahul Jain, Jingyu Shi, VK Chaitanya, Hyung-Gun Chi, Subramanian Chidambaram, Alexander Quinn, Karthik Ramani
IEEE Robotics and Automation Letters, vol. 9, no. 1, pp. 451-458, Jan. 2024

Interacting Objects: A dataset of object-object interactions for richer dynamic scene representations Asim Unmesh, Rahul Jain, Jingyu Shi, V. K. Chaithanya Manam, Hyung-Gun Chi, Subramanian Chidambaram, Alexander J. Quinn, Karthik Ramani IEEE...

An HCI-Centric Survey and Taxonomy of Human-Generative-AI Interactions

An HCI-Centric Survey and Taxonomy of Human-Generative-AI Interactions

Jingyu Shi*, Rahul Jain*, Hyungjun Doh, Ryo Suzuki, Karthik Ramani
Submitted to 2024 CHI Conference on Human Factors in Computing Systems

Generative AI (GenAI) has shown remarkable capabilities in generating diverse and realistic content across different formats like images, videos, and text. In Generative AI, human involvement is essential, thus HCI literature has investigated how...

AdamsFormer for Spatial Action Localization in the Future

AdamsFormer for Spatial Action Localization in the Future

Hyung-Gun Chi, Kwonjoon Lee, Nakul Agarwal, Yi Xu, Karthik Ramani, Chiho Choi
2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

Predicting future action locations is vital for applications like human-robot collaboration. While some computer vision tasks have made progress in predicting human actions, accurately localizing these actions in future frames remains an area with...

Ubi Edge: Authoring Edge-Based Opportunistic Tangible User Interfaces in Augmented Reality

Ubi Edge: Authoring Edge-Based Opportunistic Tangible User Interfaces in Augmented Reality

Fengming He, Xiyun Hu, Jingyu Shi, Xun Qian, Tianyi Wang, Karthik Ramani
In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems

Edges are one of the most ubiquitous geometric features of physical objects. They provide accurate haptic feedback and easy-totrack features for camera systems, making them an ideal basis for Tangible User Interfaces (TUI) in Augmented Reality...

InstruMentAR: Auto-Generation of Augmented Reality Tutorials for Operating Digital Instruments Through Recording Embodied

InstruMentAR: Auto-Generation of Augmented Reality Tutorials for Operating Digital Instruments Through Recording Embodied

Ziyi Liu, Zhengzhe Zhu, Enze Jiang, Feichi Huang, Ana Villanueva, Tianyi Wang, Xun Qian, Karthik Ramani
In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems

Augmented Reality tutorials, which provide necessary context by directly superimposing visual guidance on the physical referent, represent an effective way of scaffolding complex instrument operations. However, current AR tutorial authoring...

MechARspace: An Authoring System Enabling Bidirectional Binding of Augmented Reality with Toys in Real-time

MechARspace: An Authoring System Enabling Bidirectional Binding of Augmented Reality with Toys in Real-time

Zhengzhe Zhu, Ziyi Liu, Tianyi Wang, Youyou Zhang, Xun Qian, Pashin Farsak Raja, Ana Villanueva, Karthik Ramani
In Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology (pp. 1-16).

Augmented Reality (AR), which blends physical and virtual worlds, presents the possibility of enhancing traditional toy design. By leveraging bidirectional virtual-physical interactions between humans and the designed artifact, such AR-enhanced...

ARnnotate: An Augmented Reality Interface for Collecting Custom Dataset of 3D Hand-Object Interaction Pose Estimation

ARnnotate: An Augmented Reality Interface for Collecting Custom Dataset of 3D Hand-Object Interaction Pose Estimation

Xun Qian, Fengming He, Xiyun Hu, Tianyi Wang, Karthik Ramani
In Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology (pp. 1-14)

Vision-based 3D pose estimation has substantial potential in hand-object interaction applications and requires user-specified datasets to achieve robust performance. We propose ARnnotate, an Augmented Reality (AR) interface enabling end-users to...

CHIMERA: Supporting Wearables Development across Multidisciplinary Perspectives

CHIMERA: Supporting Wearables Development across Multidisciplinary Perspectives

Luis Paredes, Caroline McMillan, Wan Kyn Chan, Senthil Chandrasegaran, Ramyak Singh, Karthik Ramani, and Danielle Wilde
In the Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies

Wearable technologies draw on a range of disciplines, including fashion, textiles, HCI, and engineering. Due to differences in methodology, wearables researchers can experience gaps or breakdowns in values, goals, and vocabulary when collaborating....

GesturAR: An Authoring System for Creating Freehand Interactive Augmented Reality Applications

GesturAR: An Authoring System for Creating Freehand Interactive Augmented Reality Applications

Tianyi Wang, Xun Qian, Fengming He, Xiyun Hu, Yuanzhi Cao, Karthik Ramani
In The 34th Annual ACM Symposium on User Interface Software and Technology (UIST '21)

Freehand gesture is an essential input modality for modern Augmented Reality (AR) user experiences. However, developing AR applications with customized hand interactions remains a challenge for end-users. Therefore, we propose GesturAR, an...

ProcessAR: An augmented reality-based tool to create in-situ procedural 2D/3D AR Instructions

ProcessAR: An augmented reality-based tool to create in-situ procedural 2D/3D AR Instructions

Subramanian Chidambaram, Hank Huang, Fengming He, Xun Qian, Ana M Villanueva, Thomas S Redick, Wolfgang Stuerzlinger, Karthik Ramani
In Proceedings of the Designing Interactive Systems Conference

Augmented reality (AR) is an efficient form of delivering spatial information and has great potential for training workers. However, AR is still not widely used for such scenarios due to the technical skills and expertise required to create...

FabHandWear : An End-to-End Pipeline from Design to Fabrication of Customized Functional Hand Wearables

FabHandWear : An End-to-End Pipeline from Design to Fabrication of Customized Functional Hand Wearables

Luis Paredes, Sai Swarup Reddy, Subramanian Chidambaram, Devashri Vagholkar, Yunbo Zhang, Bedrich Benes, Karthik Ramani
In Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies

Current hand wearables have limited customizability, they are loose-fit to an individual's hand and lack comfort. The main barrier in customizing hand wearables is the geometric complexity and size variation in hands. Moreover, there are different...

RobotAR: An Augmented Reality Compatible Teleconsulting Robotics Toolkit for Augmented Makerspace Experiences

RobotAR: An Augmented Reality Compatible Teleconsulting Robotics Toolkit for Augmented Makerspace Experiences

Ana Villanueva, Zhengzhe Zhu, Ziyi Liu, Xin Du, Joey Huang, Kylie Peppler, Karthik Ramani
In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems

Distance learning is facing a critical moment finding a balance between high quality education for remote students and engaging them in hands-on learning. This is particularly relevant for project-based classrooms and makerspaces, which typically...

CAPturAR: An Augmented Reality Tool for Authoring Human-Involved Context-Aware Applications

CAPturAR: An Augmented Reality Tool for Authoring Human-Involved Context-Aware Applications

Tianyi Wang*, Xun Qian*, Fengming He, Xiyun Hu, Ke Huo, Yuanzhi Cao, Karthik Ramani
In Proceedings of the 2020 UIST 33rd ACM User Interface Software and Technology Symposium

Recognition of human behavior plays an important role in context-aware applications. However, it is still a challenge for end-users to build personalized applications that accurately recognize their own activities. Therefore, we present CAPturAR,...

An Exploratory Study of Augmented Reality Presence for Tutoring Machine Tasks

An Exploratory Study of Augmented Reality Presence for Tutoring Machine Tasks

Yuanzhi Cao, Xun Qian, Tianyi Wang, Rachel Lee, Ke Huo, Karthik Ramani
In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems

Machine tasks in workshops or factories are often a compound sequence of local, spatial, and body-coordinated human-machine interactions. Prior works have shown the merits of video-based and augmented reality (AR) tutoring systems for local tasks....

Meta-AR-App: An Authoring Platform for Collaborative Augmented Reality in STEM Classrooms

Meta-AR-App: An Authoring Platform for Collaborative Augmented Reality in STEM Classrooms

Ana Villanueva, Zhengzhe Zhu, Ziyi Liu, Thomas Redick, Kylie Peppler, and Karthik Ramani
In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems

Augmented Reality (AR) has become a valuable tool for education and training processes. Meanwhile, cloud-based technologies can foster collaboration and other interaction modalities to enhance learning. We combine the cloud capabilities with AR...

GhostAR: A Time-space Editor for Embodied Authoring of Human-Robot Collaborative Task with Augmented Reality

GhostAR: A Time-space Editor for Embodied Authoring of Human-Robot Collaborative Task with Augmented Reality

Yuanzhi Cao*, Tianyi Wang*, Xun Qian, Pawan S. Rao, Manav Wadhawan, Ke Huo, Karthik Ramani
Proceedings of the 32nd Annual Symposium on User Interface Software and Technology. ACM, 2019.

We present GhostAR, a time-space editor for authoring and acting Human-Robot-Collaborative (HRC) tasks in-situ. Our system adopts an embodied authoring approach in Augmented Reality (AR), for spatially editing the actions and programming the robots...

V.Ra: An In-Situ Visual Authoring System for Robot-IoT Task Planning with Augmented Reality

V.Ra: An In-Situ Visual Authoring System for Robot-IoT Task Planning with Augmented Reality

Yuanzhi Cao, Zhuangying Xu, Fan Li, Wentao Zhong, Ke Huo, and Karthik Ramani
In Proceedings of the 2019 on Designing Interactive Systems Conference (pp. 1059-1070). ACM.

We present V.Ra, a visual and spatial programming system for robot-IoT task authoring. In V.Ra, programmable mobile robots serve as binding agents to link the stationary IoTs and perform collaborative tasks. We establish an ecosystem that...

Shape Structuralizer: Design, Fabrication, and User-driven Iterative Refinement of 3D Mesh Models

Shape Structuralizer: Design, Fabrication, and User-driven Iterative Refinement of 3D Mesh Models

Subramanian Chidambaram, Yunbo Zhang, Venkatraghavan Sundararajan, Niklas Elmqvist, and Karthik Ramani
ACM SIGCHI Conference on Human Factors in Computing Systems (CHI 2019), May 4–9, 2019, Glasgow, Scotland, UK

Current Computer-Aided Design (CAD) tools lack proper support for guiding novice users towards designs ready for fabrication. We propose Shape Structuralizer (SS), an interactive design support system that repurposes surface models into structural...

Scenariot: Spatially Mapping Smart Things Within Augmented Reality Scenes

Scenariot: Spatially Mapping Smart Things Within Augmented Reality Scenes

Ke Huo, Yuanzhi Cao, Sang Ho Yoon, Zhuangying Xu, Guiming Chen, Karthik Ramani
Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, CHI 2018, April 21–26, 2018, Montreal, QC, Canada

The emerging simultaneous localizing and mapping (SLAM) based tracking technique allows the mobile AR device spatial awareness of the physical world. Still, smart things are not fully supported with the spatial awareness in AR. Therefore, we...

iSoft: A Customizable Soft Sensor with Real-time Continuous Contact and Stretching Sensing

iSoft: A Customizable Soft Sensor with Real-time Continuous Contact and Stretching Sensing

Sang Ho Yoon, Ke Huo, Yunbo Zhang, Guiming Chen, Luis Paredes, Subramanian Chidambaram, Karthik Ramani
In Proceedings of the 30th Annual ACM Symposium on User Interface Software & Technology (UIST'17) , Quebec City, Canada, 2017 (Acceptance Rate: 22.5%)

We present iSoft, a single volume soft sensor capable of sensing real-time continuous contact and unidirectional stretching. We propose a low-cost and an easy way to fabricate such piezoresistive elastomer-based soft sensors for instant...

RevoMaker: Enabling Multi-directional and Functionally-embedded 3D Printing using a Rotational Cuboidal Platform

RevoMaker: Enabling Multi-directional and Functionally-embedded 3D Printing using a Rotational Cuboidal Platform

Wei Gao, Yunbo Zhang, Diogo C. Nazzetta, Karthik Ramani, Raymond J. Cipra
In Proceedings of ACM Symposium on User Interface Software and Technology, (UIST '15), Charlotte, NC, USA

In recent years, 3D printing has gained significant attention from the maker community, academia, and industry to support low-cost and iterative prototyping of designs. Current unidirectional extrusion systems require printing sacrificial material...

Sketcholution: Interaction Histories for Sketching

Sketcholution: Interaction Histories for Sketching

Zhenpeng Zhao, William Benjamin, Niklas Elmqvist, Karthik Ramani
International Journal of Human-Computer Studies, 82 (2015) 11-20

We present Sketcholution, a method for automatically creating visual histories of hand-drawn sketches. Such visual histories are useful for a designer to reflect on a sketch, communicate ideas to others, and fork from or revert to an earlier point...

Integrated Sustainable Lifecycle Design: A Review

Integrated Sustainable Lifecycle Design: A Review

Karthik Ramani, Devarajan Ramanujan, William Z Bernstein, Fu Zhao, John Sutherland, Carol Handwerker, Jun-Ki Choi, Harrison Kim, and Deborah Thurston
Published in the Journal of Mechanical Design, Sept. 2010, Vol. 132, No. 9, pp. 1-15.

Abstract:  Product design is one of the most important sectors influencing global sustainability, as almost all the products consumed by people are outputs of the product development process. In particular, early design decisions can have a very...