AnnotateXR: An Extended Reality Workflow for Automating Data Annotation to Support Computer Vision Applications

AnnotateXR: An Extended Reality Workflow for Automating Data Annotation to Support Computer Vision Applications

Subramanian Chidambaram*, Rahul Jain*, Sai Swarup Reddy, Asim Unmesh, Karthik Ramani
J. Comput. Inf. Sci. Eng. Dec 2024, 24(12): 121001 (13 pages)

Computer vision (CV) algorithms require large annotated datasets that are often labor-intensive and expensive to create. We propose AnnotateXR, an extended reality (XR) workflow to collect various high-fidelity data and auto-annotate it in a single...

Multi-Modal Representation Learning with Tactile Data

Multi-Modal Representation Learning with Tactile Data

Hyung-Gun Chi, Jose Barreiros, Jean Mercat, Karthik Ramani, Thomas Kollar
In 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

Advancements in embodied language models like PALM-E and RT-2 have significantly enhanced language-conditioned robotic manipulation. However, these advances remain predominantly focused on vision and language, often overlooking the pivotal role of...

Estimating Ego-Body Pose from Doubly Sparse Egocentric Video Data, Advances in Neural Information Processing Systems

Estimating Ego-Body Pose from Doubly Sparse Egocentric Video Data, Advances in Neural Information Processing Systems

Seunggeun Chi, Pin-Hao Huang, Enna Sachdeva, Hengbo Ma, Karthik Ramani, Kwonjoon Lee
Conference on Neural Information Processing Systems (NeurIPS 2024)

We study the problem of estimating the body movements of a camera wearer from egocentric videos. Current methods for ego-body pose estimation rely on temporally dense sensor data, such as IMU measurements from spatially sparse body parts like the...

M2D2M: Multi-Motion Generation from Text with Discrete Diffusion Models

M2D2M: Multi-Motion Generation from Text with Discrete Diffusion Models

Seunggeun Chi*, Hyung-gun Chi*, Hengbo Ma, Nakul Agarwal, Faizan Siddiqui, Karthik Ramani, Kwonjoon Lee
In European Conference on Computer Vision, 2024.

We introduce the Multi-Motion Discrete Diffusion Models (M2D2M), a novel approach for human motion generation from textual descriptions of multiple actions, utilizing the strengths of discrete diffusion models. This approach adeptly addresses the...

ClassMeta: Designing Interactive Virtual Classmate to Promote VR Classroom Participation

ClassMeta: Designing Interactive Virtual Classmate to Promote VR Classroom Participation

Ziyi Liu*, Zhengzhe Zhu*, Lijun Zhu, Enze Jiang, Xiyun Hu, Kylie A Peppler, Karthik Ramani
In Proceedings of the CHI Conference on Human Factors in Computing Systems, pp. 1-17. 2024.

Peer influence plays a crucial role in promoting classroom participation, where behaviors from active students can contribute to a collective classroom learning experience. However, the presence of these active students depends on several...