Human-Machine Interaction Assessment by Neurophysiological Measures: A Study on Professional Air Traffic Controllers

Aricò, Pietro; Reynal, Maxime; Imbert, Jean-Paul; Hurter, Christophe; Borghini, Gianluca; Flumeri, Gianluca Di; Sciaraffa, Nicolina; Florio, Antonio Di; Terenzi, Michela; Ferreira, Ana; Pozzi, Simone; Betti, Viviana; Marucci, Matteo; Pavone, Enea Francesco; Telea, Alexandru; Babiloni, Fabio

doi:10.1109/embc.2018.8513212

Cited by 15 publications

(13 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Air traffic control has been classified by the U.S. 20,21 . The majority of studies related to ATM contexts are focused on task complexity and mental workload rather than investigating the impacts and consequences of stress on controllers [22][23][24][25][26][27][28][29][30][31][32][33][34] . Stress responses are characterised by the onset of mental and physical alterations.…”

Section: Neurophysiology Of Stressmentioning

confidence: 99%

A multimodal and signals fusion approach for assessing the impact of stressful events on Air Traffic Controllers

Borghini

Flumeri

Aricò

et al. 2020

Sci Rep

Self Cite

View full text Add to dashboard Cite

Stress is a word used to describe human reactions to emotionally, cognitively and physically challenging experiences. A hallmark of the stress response is the activation of the autonomic nervous system, resulting in the "fight-freeze-flight" response to a threat from a dangerous situation. Consequently, the capability to objectively assess and track a controller's stress level while dealing with air traffic control (ATC) activities would make it possible to better tailor the work shift and maintain high safety levels, as well as to preserve the operator's health. In this regard, sixteen controllers were asked to perform a realistic air traffic management (ATM) simulation during which subjective data (i.e. stress perception) and neurophysiological data (i.e. brain activity, heart rate, and galvanic skin response) were collected with the aim of accurately characterising the controller's stress level experienced in the various experimental conditions. In addition, external supervisors regularly evaluated the controllers in terms of manifested stress, safety, and efficiency throughout the ATM scenario. The results demonstrated 1) how the stressful events caused both supervisors and controllers to underestimate the experienced stress level, 2) the advantage of taking into account both cognitive and hormonal processes in order to define a reliable stress index, and 3) the importance of the points in time at which stress is measured owing to the potential transient effect once the stressful events have ceased. Stress in air traffic management. The air traffic management (ATM) system is currently going through a major restructuring process, covering the entire architecture of process, personnel, and management environments. The increase in the level of automation as support tools for air traffic control (ATC) activities is one of the main technological challenges which will be faced in the future 1,2. Automation, digitalisation of information and virtualisation of the ATC operations will require a transformation of tasks and working methods. For example, with the prospect of an increasing in the volume of air traffic to be managed, controllers will be expected to delegate more activities and decisions to automation. In ATC, high demand is usually associated with a high mental workload and level of responsibility, while stress is linked to available time, team support, adherence to procedures and control over the planning and execution of tasks. The concepts of mental workload and stress are often used ambiguously, especially because of their relationship with performance when they reach very low or high values. In such cases, performance drops dramatically. However, certain levels of mental workload and

show abstract

Section: Neurophysiology Of Stressmentioning

confidence: 99%

A multimodal and signals fusion approach for assessing the impact of stressful events on Air Traffic Controllers

Borghini

Flumeri

Aricò

et al. 2020

Sci Rep

Self Cite

View full text Add to dashboard Cite

show abstract

“…Therefore, to generalize our results and further build a generic regression model to predict the dynamic ranges of EEG features, larger EEG datasets need to be employed. Second, validation of the regression model with totally new EEG datasets recorded by using different task paradigms would help to precisely evaluate the performance of our approach [38][39][40][41]. Third, the threshold value to determine whether a participant is appropriate for pBCI needs to be determined empirically considering the actual pBCI users' satisfaction level.…”

Section: Discussionmentioning

confidence: 99%

Prediction of Individual User’s Dynamic Ranges of EEG Features from Resting-State EEG Data for Evaluating Their Suitability for Passive Brain–Computer Interface Applications

Cha

Han

2020

Sensors

View full text Add to dashboard Cite

With the recent development of low-cost wearable electroencephalogram (EEG) recording systems, passive brain–computer interface (pBCI) applications are being actively studied for a variety of application areas, such as education, entertainment, and healthcare. Various EEG features have been employed for the implementation of pBCI applications; however, it is frequently reported that some individuals have difficulty fully enjoying the pBCI applications because the dynamic ranges of their EEG features (i.e., its amplitude variability over time) were too small to be used in the practical applications. Conducting preliminary experiments to search for the individualized EEG features associated with different mental states can partly circumvent this issue; however, these time-consuming experiments were not necessary for the majority of users whose dynamic ranges of EEG features are large enough to be used for pBCI applications. In this study, we tried to predict an individual user’s dynamic ranges of the EEG features that are most widely employed for pBCI applications from resting-state EEG (RS-EEG), with the ultimate goal of identifying individuals who might need additional calibration to become suitable for the pBCI applications. We employed a machine learning-based regression model to predict the dynamic ranges of three widely used EEG features known to be associated with the brain states of valence, relaxation, and concentration. Our results showed that the dynamic ranges of EEG features could be predicted with normalized root mean squared errors of 0.2323, 0.1820, and 0.1562, respectively, demonstrating the possibility of predicting the dynamic ranges of the EEG features for pBCI applications using short resting EEG data.

show abstract

“…Although many works have demonstrated the possibility to measure user’s mental states in laboratory settings by using neurophysiological measures, just few works demonstrated the applicability of neurophysiological measures out of the labs i.e., real/realistic settings (Aloise et al, 2013; Cartocci et al, 2015; Aricò et al, 2016a, 2017a, 2018b; Blankertz et al, 2016; Vecchiato et al, 2016; Borghini et al, 2017a, b; Cartocci et al, 2018a,b, 2019a,b; Di Flumeri et al, 2018; Modica et al, 2017, 2018). Even less, the studies demonstrating the effectiveness of such measures in comparing different solutions in operational environments.…”

Section: Introductionmentioning

confidence: 99%

“…In this regard, Borghini et al (2015) performed a pilot study on professional helicopter pilots performing simulated missions carried out at AgustaWestland facilities in Yeovil (UK) with the aim to compare different avionic technologies not only in terms of performance, but also regarding the experienced workload during their use by using an EEG-based index. Aricò et al (2018b) proposed a work aiming at investigating the possibility of employing neurophysiological-based measures to assess the human-machine interaction effectiveness. In particular, two different interaction modalities (normal and augmented) related to the Air Traffic Management (ATM) field have been compared in terms of behavioral performance and an EEG-based workload index (frontal theta and parietal alpha ratio), by involving professional ATCOs in a control tower simulated environment at ENAC, Toulouse (France) in a laboratory-based task.…”

Section: Introductionmentioning

confidence: 99%

How Neurophysiological Measures Can be Used to Enhance the Evaluation of Remote Tower Solutions

Aricò

Reynal

Flumeri

et al. 2019

Front. Hum. Neurosci.

Self Cite

View full text Add to dashboard Cite

New solutions in operational environments are often, among objective measurements, evaluated by using subjective assessment and judgment from experts. Anyhow, it has been demonstrated that subjective measures suffer from poor resolution due to a high intra and inter-operator variability. Also, performance measures, if available, could provide just partial information, since an operator could achieve the same performance but experiencing a different workload. In this study, we aimed to demonstrate: (i) the higher resolution of neurophysiological measures in comparison to subjective ones; and (ii) how the simultaneous employment of neurophysiological measures and behavioral ones could allow a holistic assessment of operational tools. In this regard, we tested the effectiveness of an electroencephalography (EEG)-based neurophysiological index (WEEG index) in comparing two different solutions (i.e., Normal and Augmented) in terms of experienced workload. In this regard, 16 professional air traffic controllers (ATCOs) have been asked to perform two operational scenarios. Galvanic Skin Response (GSR) has also been recorded to evaluate the level of arousal (i.e., operator involvement) during the two scenarios execution. NASA-TLX questionnaire has been used to evaluate the perceived workload, and an expert was asked to assess performance achieved by the ATCOs. Finally, reaction times on specific operational events relevant for the assessment of the two solutions, have also been collected. Results highlighted that the Augmented solution induced a local increase in subjects performance (Reaction times). At the same time, this solution induced an increase in the workload experienced by the participants (WEEG). Anyhow, this increase is still acceptable, since it did not negatively impact the performance and has to be intended only as a consequence of the higher engagement of the ATCOs. This behavioral effect is totally in line with physiological results obtained in terms of arousal (GSR), that increased during the scenario with augmentation. Subjective measures (NASA-TLX) did not highlight any significant variation in perceived workload. These results suggest that neurophysiological measure provide additional information than behavioral and subjective ones, even at a level of few seconds, and its employment during the pre-operational activities (e.g., design process) could allow a more holistic and accurate evaluation of new solutions.

show abstract

Human-Machine Interaction Assessment by Neurophysiological Measures: A Study on Professional Air Traffic Controllers

Cited by 15 publications

References 11 publications

A multimodal and signals fusion approach for assessing the impact of stressful events on Air Traffic Controllers

A multimodal and signals fusion approach for assessing the impact of stressful events on Air Traffic Controllers

Prediction of Individual User’s Dynamic Ranges of EEG Features from Resting-State EEG Data for Evaluating Their Suitability for Passive Brain–Computer Interface Applications

How Neurophysiological Measures Can be Used to Enhance the Evaluation of Remote Tower Solutions

Contact Info

Product

Resources

About