Adaptive and Multimodal Transparency for Real-Time Collaborative Human-Machine Interactions

Interdisciplinary Areas: Autonomous and Connected Systems, Human-Machine/Computer Interaction, Human Factors, Human-Centered Design

Project Description

It is well-established that cognitive factors, including the human’s trust and workload, affect decision-making and performance during interactions with autonomous systems. Given this, adaptive automation, in which an autonomous system is able to sense and respond to changes in various cognitive factors, holds promise for mitigating pitfalls associated with human misuse, disuse, and abuse of automation. However, accurately “sensing” the cognitive state of the human and contextually “responding” via visual, auditory, and/or tactile sensory communication, remains a challenge for effective human-automation teaming.

The objective of this research project is to exploit heterogenous human data types (i.e., behavioral, psychophysiological, and self-report) for the purpose of designing real-time control strategies for adaptive and multimodal transparency that account for trust in automation, situational awareness, and workload within human-machine teaming contexts. The postdoctoral scholar will be involved in human-subject experiment ideation as well as design, data collection, processing, and analysis. A core component of the research will be on sensor fusion for real-time human cognitive state estimation.

Start Date


Postdoctoral Qualifications

Strong analytical skills in signal processing, control, and optimization. Experience or expertise designing human-subject experiments. Passion for, and interest in, solving challenging research problems using methodologies from different areas. Good oral and written communication skills in English. Ability to thrive in a collaborative environment.


Brandon Pitts,, School of Industrial Engineering,

Neera Jain,, School of Mechanical Engineering,

Center for Innovation in Control, Optimization, and Networks (ICON):

External Collaborators

Robert Proctor, Department of Psychological Sciences (Purdue),


Huang & Pitts (2021). The Effects of Age and Physical Exercise on Multimodal Signal Perception: Implications for Semi-autonomous Vehicle Takeover Requests. Accepted in Applied Ergonomics.

Liang, N., Yang, J., Yu, D., Prakah-Asante, K. O., Curry, R., Blommer, M., Swaminathan, R. & Pitts, B. J. (2021). Using eye-tracking to investigate the effects of pre-takeover visual engagement on situation awareness during automated driving. Accident Analysis & Prevention, 157, 106143.

Sheridan, T. B. (2011). Adaptive automation, level of automation, allocation authority, supervisory control, and adaptive control: Distinctions and modes of adaptation. IEEE Transactions on Systems, Man, and Cybernetics-Part A: Systems and Humans, 41(4), 662-667.

K. Akash, W.-L. Hu, N. Jain, and T. Reid, “A Classification Model for Sensing Human Trust in Machines Using EEG and GSR,” ACM Transactions on Interactive Intelligent Systems, vol. 8, no. 4, pp. 1-20, Nov. 2018, doi: 10.1145/3132743

K. Akash, G. McMahon, T. Reid, and N. Jain, “Human Trust-based Feedback Control: Dynamically varying automation transparency to optimize human-machine interactions.” IEEE Control Systems Magazine, vol. 40, no. 6, pp. 98-116, Dec. 2020, doi: 10.1109/MCS.2020.3019151