Modeling Cumulative Arm Fatigue in Mid-Air Interaction based on Perceived Exertion and Kinetics of Arm Motion

by | Jan 14, 2017

Authors: Sujin Jang, Wolfgang Stuerzlinger, Satyajit Ambike, Karthik Ramani
In Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI 2017: 3328-3339), Denver, CO, May 6-11, 2017 (Acceptance Rate: 25%)

Estimating subjective fatigue based on cumulative fatigue model and biomechanical upper limb analysis in mid-air interaction. Left: sketches of mid-air pointing tasks performed in the experiments. Middle: biomechanical model of the upper limb and three-compartment muscle (TCM) model. Right: results of the leave-one-out cross-validation of the TCM model across all subject data (green-upward/downward triangles: upper/lower bound of ground-truth, blue circles: averaged ground-truth, black crosses: TCM estimates, red circles: averaged TCM estimages, orange/purple circles: averaged existing fatigue metric).

Quantifying cumulative arm muscle fatigue is a critical factor in understanding, evaluating, and optimizing user experience during prolonged mid-air interaction. A reasonably accurate estimation of fatigue requires an estimate of an individual’s strength. However, there is no easy-to-access method to measure individual strength to accommodate inter-individual differences. Furthermore, fatigue is influenced by both psychological and physiological factors, but no current HCI model provides good estimates of cumulative subjective fatigue. We present a new, simple method to estimate the maximum shoulder torque through a mid-air pointing task, which agrees with direct strength measurements. We then introduce a cumulative fatigue model informed by subjective and biomechanical measures. We evaluate the performance of the model in estimating cumulative subjective fatigue in mid-air interaction by performing multiple cross-validations and a comparison with an existing fatigue metric. Finally, we discuss the potential of our approach for real-time evaluation of subjective fatigue as well as future challenges.

We released the fatigue model implementation and the biomechanical upper limb analysis [GitHub Link].



Sujin Jang is currently working at Motorola, Chicago, IL. He received his Ph.D. from the School of Mechanical Engineering at Purdue University in August 2017. His research work at the C-Design Lab broadly involved human-computer interaction, visual analytics, machine learning, and robotics. His research has focused on creating methodologies and principles for effective use of gestures in HCI. In particular, he has developed methods to analyze and exploit human gesture based on visual analytics integrating machine learning and information visualization; biomechanical arm fatigue analysis; a gestural user interface for human-robot interaction; and an interactive clustering and collaborative filtering approach for hand pose estimation. He also has served as a teaching assistant for ME 444: Computer-aided design and rapid prototyping, and received the Estus H. and Vashti L. Magoon Award for Teaching Excellence in 2015. [Personal Website][LinkedIn]