A Neuroadaptive Cognitive Model for Dealing With Uncertainty in Tracing Pilots' Cognitive State

Klaproth, Oliver W.; Halbrügge, Marc; Krol, Laurens R.; Vernaleken, Christoph; Zander, Thorsten O.; Rußwinkel, Nele

doi:10.1111/tops.12515

Cited by 33 publications

(21 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Historically, different modalities have also been used by different researchers that play a crucial role in exploring successful HMT implementation. Researchers from [21] used a model-based approach for cognitive assistance to track pilots' changing demands in a dynamic situation. They used flight deck interactions and EEG recordings to find individual pilot's behavior in response to flight deck alerts.…”

Section: Background and Literature Reviewmentioning

confidence: 99%

Novel Human-in-the-Loop (HIL) Simulation Method to Study Synthetic Agents and Standardize Human–Machine Teams (HMT)

Damacharla¹,

Dhakal

Bandreddi

et al. 2020

Applied Sciences

View full text Add to dashboard Cite

This work presents a multi-year study conducted at the University of Toledo, aimed at improving human–machine teaming (HMT) methods and technologies. With the advent of artificial intelligence (AI) in 21st-century machines, collaboration between humans and machines has become highly complicated for real-time applications. The penetration of intelligent and synthetic assistants (IA/SA) in virtually every field has opened up a path to the area of HMT. When it comes to crucial tasks such as patient treatment/care, industrial production, and defense, the use of non-standardized HMT technologies may pose a risk to human lives and cost billions of taxpayer dollars. A thorough literature survey revealed that there are not many established standards or benchmarks for HMT. In this paper, we propose a method to design an HMT based on a generalized architecture. This design includes the development of an intelligent collaborative system and the human team. Followed by the identification of processes and metrics to test and validate the proposed model, we present a novel human-in-the-loop (HIL) simulation method. The effectiveness of this method is demonstrated using two controlled HMT scenarios: Emergency care provider (ECP) training and patient treatment by an experienced medic. Both scenarios include humans processing visual data and performing actions that represent real-world applications while responding to a Voice-Based Synthetic Assistant (VBSA) as a collaborator that keeps track of actions. The impact of various machines, humans, and HMT parameters is presented from the perspective of performance, rules, roles, and operational limitations. The proposed HIL method was found to assist in standardization studies in the pursuit of HMT benchmarking for critical applications. Finally, we present guidelines for designing and benchmarking HMTs based on the case studies’ results analysis.

show abstract

Section: Background and Literature Reviewmentioning

confidence: 99%

Novel Human-in-the-Loop (HIL) Simulation Method to Study Synthetic Agents and Standardize Human–Machine Teams (HMT)

Damacharla¹,

Dhakal

Bandreddi

et al. 2020

Applied Sciences

View full text Add to dashboard Cite

show abstract

Section: Methodsmentioning

confidence: 99%

“…The distribution of lacking and inadequate responses was tested for a relationship with EEG classifications by a Chi-square test. A detailed description of the cognitive model including the overall approach and modeling decisions made can be found in Klaproth et al (2020). Figure 3 shows the grand-average ERPs on channel Pz for the standard and target tones during the oddball experiment on three electrode sites.…”

Section: Cognitive Modelmentioning

confidence: 99%

Tracing Pilots’ Situation Assessment by Neuroadaptive Cognitive Modeling

Klaproth

Vernaleken

Krol³

et al. 2020

Front. Neurosci.

Self Cite

View full text Add to dashboard Cite

This study presents the integration of a passive brain-computer interface (pBCI) and cognitive modeling as a method to trace pilots' perception and processing of auditory alerts and messages during operations. Missing alerts on the flight deck can result in out-of-the-loop problems that can lead to accidents. By tracing pilots' perception and responses to alerts, cognitive assistance can be provided based on individual needs to ensure they maintain adequate situation awareness. Data from 24 participating aircrew in a simulated flight study that included multiple alerts and air traffic control messages in single pilot setup are presented. A classifier was trained to identify pilots' neurophysiological reactions to alerts and messages from participants' electroencephalogram (EEG). A neuroadaptive ACT-R model using EEG data was compared to a conventional normative model regarding accuracy in representing individual pilots. Results show that passive BCI can distinguish between alerts that are processed by the pilot as task-relevant or irrelevant in the cockpit based on the recorded EEG. The neuroadaptive model's integration of this data resulted in significantly higher performance of 87% overall accuracy in representing individual pilots' responses to alerts and messages compared to 72% accuracy of a normative model that did not consider EEG data. We conclude that neuroadaptive technology allows for implicit measurement and tracing of pilots' perception and processing of alerts on the flight deck. Careful handling of uncertainties inherent to passive BCI and cognitive modeling shows how the representation of pilot cognitive states can be improved iteratively for providing assistance.

show abstract

“…This shows the importance of research on cognitive modeling approaches that enable robots or intelligent systems to gain an understanding of the human partner (Kambhampati, 2019;Klaproth et al, 2020;Rußwinkel, 2020) in order to respond to the partner comprehensively. Furthermore, the robot needs to behave in a traceable fashion, so that the human partner is motivated to help even if errors occur.…”

Section: Behavioral Reaction To Robotsmentioning

confidence: 99%

The Role of Frustration in Human–Robot Interaction – What Is Needed for a Successful Collaboration?

Weidemann

Rußwinkel

2021

Front. Psychol.

View full text Add to dashboard Cite

To realize a successful and collaborative interaction between human and robots remains a big challenge. Emotional reactions of the user provide crucial information for a successful interaction. These reactions carry key factors to prevent errors and fatal bidirectional misunderstanding. In cases where human–machine interaction does not proceed as expected, negative emotions, like frustration, can arise. Therefore, it is important to identify frustration in a human–machine interaction and to investigate its impact on other influencing factors such as dominance, sense of control and task performance. This paper presents a study that investigates a close cooperative work situation between human and robot, and explore the influence frustration has on the interaction. The task for the participants was to hand over colored balls to two different robot systems (an anthropomorphic robot and a robotic arm). The robot systems had to throw the balls into appropriate baskets. The coordination between human and robot was controlled by various gestures and words by means of trial and error. Participants were divided into two groups, a frustration- (FRUST) and a no frustration- (NOFRUST) group. Frustration was induced by the behavior of the robotic systems which made errors during the ball handover. Subjective and objective methods were used. The sample size of participants was N = 30 and the study was conducted in a between-subject design. Results show clear differences in perceived frustration in the two condition groups and different behavioral interactions were shown by the participants. Furthermore, frustration has a negative influence on interaction factors such as dominance and sense of control. The study provides important information concerning the influence of frustration on human–robot interaction (HRI) for the requirements of a successful, natural, and social HRI. The results (qualitative and quantitative) are discussed in favor of how a successful und effortless interaction between human and robot can be realized and what relevant factors, like appearance of the robot and influence of frustration on sense of control, have to be regarded.

show abstract

A Neuroadaptive Cognitive Model for Dealing With Uncertainty in Tracing Pilots' Cognitive State

Cited by 33 publications

References 23 publications

Novel Human-in-the-Loop (HIL) Simulation Method to Study Synthetic Agents and Standardize Human–Machine Teams (HMT)

Novel Human-in-the-Loop (HIL) Simulation Method to Study Synthetic Agents and Standardize Human–Machine Teams (HMT)

Tracing Pilots’ Situation Assessment by Neuroadaptive Cognitive Modeling

The Role of Frustration in Human–Robot Interaction – What Is Needed for a Successful Collaboration?

Contact Info

Product

Resources

About