Sujin Jang, Niklas Elmqvist, and Karthik Ramani
MotionFlow: Visual Abstraction and Aggregation of Sequential Patterns in Human Motion Tracking Data
IEEE Transaction on Visualization and Computer Graphics (TVCG) vol.22, no.1, pp.21-30, Jan. 31, 2016 / IEEE Conference on Visual Analytics Science and Technology (VAST) Oct. 25-30, Chicago, IL, 2015. In print.
PDF PDF

Abstract: Pattern analysis of human motions, which is useful in many research areas, requires understanding and comparison of different styles of motion patterns. However, working with human motion tracking data to support such analysis poses great challenges. In this paper, we propose MotionFlow, a visual analytics system that provides an effective overview of various motion patterns based on an interactive flow visualization. This visualization formulates a motion sequence as transitions between static poses, and aggregates these sequences into a tree diagram to construct a set of motion patterns. The system also allows the users to directly reflect the context of data and their perception of pose similarities in generating representative pose states. We provide local and global controls over the partition-based clustering process. To support the users in organizing unstructured motion data into pattern groups, we designed a set of interactions that enables searching for similar motion sequences from the data, detailed exploration of data subsets, and creating and modifying the group of motion patterns. To evaluate the usability of MotionFlow, we conducted a user study with six researchers with expertise in gesture-based interaction design. They used MotionFlow to explore and organize unstructured motion tracking data. Results show that the researchers were able to easily learn how to use MotionFlow, and the system effectively supported their pattern analysis activities, including leveraging their perception and domain knowledge.

Accepted for publication in IEEE Transaction on Visualization and Computer Graphics (TVCG)  and presentation at IEEE Conference on Visual Analytics Science and Technology (VAST) 2015 (in print)

MotionFlow for pattern analysis of human motion data. (a) Pose tree: a simplified representation of multiple motion sequences aggregating the same transitions into a tree diagram. (b) A window dedicated to show a subtree structure based on a query. (c) Space-filling treemap representation of the motion sequence data using slice-and-dice layout. (d) Node-link diagram of pose clusters (nodes) and transitions (links) between them. This view supports interactive partition-based pose clustering. (e) Multi-tab interface for storing unique motion patterns. (f) Animations of single or multiple selected human motions.

Screen capture of MotionFlow for pattern analysis of human motion data. (a) Pose tree: a simplified representation of multiple motion sequences aggregating the same transitions into a tree diagram. (b) A window dedicated to show a subtree structure based on a query. (c) Space-filling treemap representation of the motion sequence data using slice-and-dice layout. (d) Node-link diagram of pose clusters (nodes) and transitions (links) between them. This view supports interactivepartition-based pose clustering. (e) Multi-tab interface for storing unique motion patterns. (f) Animations of single or multiple selected human motions.

Sujin
About Sujin Jang

Sujin Jang received his Ph.D. in August 2017 from the School of Mechanical Engineering at Purdue University. His research work in the C-Design Lab broadly involved human-computer interaction, visual analytics, and machine learning. In particular, the goal of his research aimed (i) to create natural human-computer interaction systems using the human body as an interaction input and (ii) to establish the fundamental principles when designing such novel interaction systems. His past research experience also includes computer vision, robotics, and non-linear control/estimation. He has received the Estus H. and Vashti L. Magoon Award for Teaching Excellence in 2015.  Currently, he is a staff research member at the Motorola-Lenovo Research group. [Personal Website][LinkedIn]

Publications
Tagged with:
Posted in 2015, HUMAN SHAPE INTERACTION, Karthik Ramani, Mid-air Interaction, Spatial Analytics, Sujin Jang