Machine-Learning Based Monitoring of Cognitive Workload in Rescue Missions With Drones

Abstract

In search and rescue missions, drone operations are challenging and cognitively demanding. High levels of cognitive workload can affect rescuers' performance, leading to failure with catastrophic outcomes. To face this problem, we propose a machine learning algorithm for real-time cognitive workload monitoring to understand if a search and rescue operator has to be replaced or if more resources are required. Our multimodal cognitive workload monitoring model combines the information of 25 features extracted from physiological signals, such as respiration, electrocardiogram, photoplethysmogram, and skin temperature, acquired in a noninvasive way. To reduce both subject and day inter-variability of the signals, we explore different feature normalization techniques, and introduce a novel weighted-learning method based on support vector machines suitable for subject-specific optimizations. On an unseen test set acquired from 34 volunteers, our proposed subject-specific model is able to distinguish between low and high cognitive workloads with an average accuracy of 87.3% and 91.2% while controlling a drone simulator using both a traditional controller and a new-generation controller, respectively.

An Overview of Approaches and Methods for the Cognitive Workload Estimation in Human–Machine Interaction Scenarios through Wearables Sensors

“…Dell'Agnola et al [88] proposed a new weighted-learning method for Support Vector Machine (SVM) to optimize the model for specific subjects. The study discussed the selection of ML algorithms based on factors such as data size and system requirements, with SVM being commonly used, but not consistently indicated as the best model.…”

Next Generation Computing and Communication Hub for First Responders in Smart Cities

“…There are various sources to create a composite NGFR platform. Among first responder platforms such as [6][7][8]18,42], we narrowed down these two: (1) WiLIFE computing platform [8] and (2) NGFR SmartHub platform [7]. Both platforms can be defined as furthergeneration computing architectures for emergency response in a smart city, both extensively use distributed resources.…”

“…We reported the results on the formalization of trust, risk, and bias assessment in such teams in our previous paper [112]. Future first responders will face the challenges of multi-task operation and simultaneous usage of various supporting resources [11,113], building trust in AI-enabled tools by allowing the tracking of risks and biases associated with this trust [112,114,115], as well as Cognitive Workload (CW) monitoring and prediction of the human-robot teams [18][19][20][21], including stress control and prediction [24,25]. − Human digital twins seem to be the key driver and framework of application AIenabled tools.…”

“…To carry out these and other emergency and rescue tasks, an on-body computation and communication network must be created. A central part of the on-body network includes AI-based tools for the responders' Cognitive Workload (CW) monitoring and prediction [18][19][20][21], health and emotional state [9,22,23], and stress control and prediction [24,25].…”

“…Public area networks for first responder needs that are provided by smart city infrastructure, e.g., [6,8,42], including big data stream [2,13,43] and unmanned aerial surveillance [18,44]; 2.…”