Towards a multimodal bioelectrical framework for the online mental workload evaluation

Aricò, Pietro; Borghini, Gianluca; Graziani, Ilenia; Taya, Fumihico; Sun, Yu; Bezerianos, Anastasios; Thakor, Nitish V.; Cincotti, Febo; Babiloni, Fabio

doi:10.1109/embc.2014.6944254

Cited by 32 publications

(25 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Similar research has been conducted in other fields, such as HCI, driving, and flight control. The performance trends of individual sensors in this study were aligned with the findings from other areas, such as driving [87,88], flight control [90], first-person shooting simulation [95], and so on. When setting up the workload prediction algorithm, the specific domain information was not used explicitly in training, therefore, this approach may be generalizable to other domains.…”

Section: Model Generalizationssupporting

confidence: 77%

See 1 more Smart Citation

Multimodal Physiological Signals for Workload Prediction in Robot-assisted Surgery

Zhou

Cha

Gonzalez

et al. 2020

J. Hum.-Robot Interact.

View full text Add to dashboard Cite

Monitoring surgeon workload during robot-assisted surgery can guide allocation of task demands, adapt system interfaces, and assess the robotic system's usability. Current practices for measuring cognitive load primarily rely on questionnaires that are subjective and disrupt surgical workflow. To address this limitation, a computational framework is demonstrated to predict user workload during telerobotic surgery. This framework leverages wireless sensors to monitor surgeons' cognitive load and predict their cognitive states. Continuous data across multiple physiological modalities (e.g., heart rate variability, electrodermal, and electroencephalogram activity) were simultaneously recorded for twelve surgeons performing surgical skills tasks on the validated da Vinci Skills Simulator. These surgical tasks varied in difficulty levels, e.g., requiring varying visual processing demand and degree of fine motor control. Collected multimodal physiological signals were fused using independent component analysis, and the predicted results were compared to the ground-truth workload level. Results compared performance of different classifiers, sensor fusion schemes, and physiological modality (i.e., prediction with single vs. multiple modalities). It was found that our multisensor approach outperformed individual signals and can correctly predict cognitive workload levels 83.2% of the time during basic and complex surgical skills tasks. CCS Concepts: • Human-centered computing → Human computer interaction (HCI); Interactive systems and tools; User interface management systems;

show abstract

Section: Model Generalizationssupporting

confidence: 77%

“…This suggests that EEG is the most predictive modality for characterizing workload levels. Other studies in domains outside of RAS [55,[87][88][89][90] also concluded that EEG was the salient modality for workload characterization. In RAS, EEG may be especially reliable due to the design of the dVSS.…”

Section: Fusionmentioning

confidence: 94%

Multimodal Physiological Signals for Workload Prediction in Robot-assisted Surgery

Zhou

Cha

Gonzalez

et al. 2020

J. Hum.-Robot Interact.

View full text Add to dashboard Cite

show abstract

“…More recently, Shou et al (2012) found that “the frontal theta EEG activity was a sensitive and reliable metric to assess workload […] during an ATC task at the resolution of minute (s).” The same findings have been highlighted by Borghini et al (2013) involving pilots in flight simulation tasks. In other recent studies involving ATCOs (Aricò et al, 2013, 2014, 2015b,c; Borghini et al, 2014; Di Flumeri et al, 2015; Toppi et al, 2016), it was demonstrated how it was possible to compute an EEG-based Workload Index able to significantly discriminate the workload demands of the ATM task, and to monitor them continuously by using frontal-parietal brain features. Other studies about the mental workload estimation by using neurophysiological indexes, have been proposed also in other operational contexts (Car drivers - Kohlmorgen et al, 2007; Borghini et al, 2012a; military domain - Dorneich et al, 2005).…”

Section: Introductionmentioning

confidence: 99%

Adaptive Automation Triggered by EEG-Based Mental Workload Index: A Passive Brain-Computer Interface Application in Realistic Air Traffic Control Environment

Aricò¹,

Borghini²,

Flumeri³

et al. 2016

Front. Hum. Neurosci.

Self Cite

190

143

View full text Add to dashboard Cite

Adaptive Automation (AA) is a promising approach to keep the task workload demand within appropriate levels in order to avoid both the under- and over-load conditions, hence enhancing the overall performance and safety of the human-machine system. The main issue on the use of AA is how to trigger the AA solutions without affecting the operative task. In this regard, passive Brain-Computer Interface (pBCI) systems are a good candidate to activate automation, since they are able to gather information about the covert behavior (e.g., mental workload) of a subject by analyzing its neurophysiological signals (i.e., brain activity), and without interfering with the ongoing operational activity. We proposed a pBCI system able to trigger AA solutions integrated in a realistic Air Traffic Management (ATM) research simulator developed and hosted at ENAC (École Nationale de l'Aviation Civile of Toulouse, France). Twelve Air Traffic Controller (ATCO) students have been involved in the experiment and they have been asked to perform ATM scenarios with and without the support of the AA solutions. Results demonstrated the effectiveness of the proposed pBCI system, since it enabled the AA mostly during the high-demanding conditions (i.e., overload situations) inducing a reduction of the mental workload under which the ATCOs were operating. On the contrary, as desired, the AA was not activated when workload level was under the threshold, to prevent too low demanding conditions that could bring the operator's workload level toward potentially dangerous conditions of underload.

show abstract

“…Between the two, the EEG is usually preferred for the workload assessment for its high temporal resolution. Moreover, it has been proved that EEG features provide higher accuracy respect to ECG and GSR ones [18], [19].…”

Section: Workload Measurementsmentioning

confidence: 99%

On the Use of Machine Learning for EEG-Based Workload Assessment: Algorithms Comparison in a Realistic Task

Sciaraffa

Aricò

Borghini

et al. 2019

Communications in Computer and Information Science

Self Cite

View full text Add to dashboard Cite

The measurement of the mental workload during real tasks by means of neurophysiological signals is still challenging. The employment of Machine Learning techniques has allowed a step forward in this direction, however, most of the work has dealt with binary classification. This study proposed to examine the surveys already performed in the context of EEG-based workload classification and to test different machine learning algorithms on real multitasking activity like the Air Traffic Management. The results obtained on 35 professional Air Traffic Controllers showed that a KNN algorithm allows discriminating up to three workload levels (low, medium and high) with more than 84 % of accuracy on average. Moreover, in such realistic employment it emerges how important is to opportunely choose the set of features to ward off that task-related confounds could affect the workload assessment.

show abstract

Towards a multimodal bioelectrical framework for the online mental workload evaluation

Cited by 32 publications

References 8 publications

Multimodal Physiological Signals for Workload Prediction in Robot-assisted Surgery

Multimodal Physiological Signals for Workload Prediction in Robot-assisted Surgery

Adaptive Automation Triggered by EEG-Based Mental Workload Index: A Passive Brain-Computer Interface Application in Realistic Air Traffic Control Environment

On the Use of Machine Learning for EEG-Based Workload Assessment: Algorithms Comparison in a Realistic Task

Contact Info

Product

Resources

About