MBioTracker: Multimodal Self-Aware Bio-Monitoring Wearable System for Online Workload Detection

Dell’Agnola, Fabio; Pale, Una; Marino, Rodrigo; Valdés, Adriana Arza; Atienza, David

doi:10.1109/tbcas.2021.3110317

Cited by 19 publications

(15 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Several studies combine physiological signals with different machine-learning algorithms for CWM in different fields [13], [14]. However, to the best of our knowledge, we are the first to address CWM of drone pilots involved in SAR missions [8], [15], [16]. Now, we extend our previous works by presenting a subject-specific CWM approach based on noninvasive physiological signals that is suitable for new drone control solutions, such as FlyJacket [17].…”

Section: Introductionmentioning

confidence: 76%

See 1 more Smart Citation

Machine-Learning Based Monitoring of Cognitive Workload in Rescue Missions With Drones

Dell’Agnola

Jao

Arza

et al. 2022

IEEE J. Biomed. Health Inform.

Self Cite

View full text Add to dashboard Cite

In search and rescue missions, drone operations are challenging and cognitively demanding. High levels of cognitive workload can affect rescuers' performance, leading to failure with catastrophic outcomes. To face this problem, we propose a machine learning algorithm for real-time cognitive workload monitoring to understand if a search and rescue operator has to be replaced or if more resources are required. Our multimodal cognitive workload monitoring model combines the information of 25 features extracted from physiological signals, such as respiration, electrocardiogram, photoplethysmogram, and skin temperature, acquired in a noninvasive way. To reduce both subject and day inter-variability of the signals, we explore different feature normalization techniques, and introduce a novel weighted-learning method based on support vector machines suitable for subject-specific optimizations. On an unseen test set acquired from 34 volunteers, our proposed subject-specific model is able to distinguish between low and high cognitive workloads with an average accuracy of 87.3% and 91.2% while controlling a drone simulator using both a traditional controller and a new-generation controller, respectively.

show abstract

Section: Introductionmentioning

confidence: 76%

“…The first preprocessing step consists of removing the artifacts from the signals with causal filters [16]. We apply a baseline wander with cutoff frequency at 0.3 Hz to both ECG and PPG signals.…”

Section: B Signal Preprocessingmentioning

confidence: 99%

Machine-Learning Based Monitoring of Cognitive Workload in Rescue Missions With Drones

Dell’Agnola

Jao

Arza

et al. 2022

IEEE J. Biomed. Health Inform.

Self Cite

View full text Add to dashboard Cite

show abstract

“…HR has also been used to detect stress levels, which exhibited correlation on subjects' stress state, regarding shortterm mental stress. It has been also used as a feature in a variety of studies addressing stress detection in humans [13,16]. In [16], a new design of a wearable embedded system for online cognitive workload monitoring is proposed.…”

Section: Human Monitoringmentioning

confidence: 99%

“…It has been also used as a feature in a variety of studies addressing stress detection in humans [13,16]. In [16], a new design of a wearable embedded system for online cognitive workload monitoring is proposed. Multiple signals are monitored (respiration (RSP), HR, skin temperature (ST) and pulse waveform) as a part of a multi-channel physiological signal acquisition and a low power processing platform.…”

Section: Human Monitoringmentioning

confidence: 99%

A real‐world data collection framework for a fused dataset creation for joint human and remotely operated vehicle monitoring and anomalous command detection

Elia

Plastiras

Pettemeridou

et al. 2022

CAAI Trans on Intel Tech

View full text Add to dashboard Cite

Unmanned Remotely Operated Vehicles (ROVs) are widely used in safety‐critical missions, spanning across monitoring, surveillance, and search and rescue missions. In most cases, the human operator controls the ROV under stressful conditions and in harsh environments. The physical and mental states of the operator are vital as they may cause involuntary movements, issuing unintentional commands due to fatigue or stress. Consequently, a joint monitoring mechanism that will assess the operator's commands can be beneficial in preventing potentially catastrophic events, such as accidents. To this end, in this study, we propose a data collection framework for creating a fused dataset, consisting of data obtained from both the human operator and the ROV, and we integrate stress induction to stimulate stress and fatigue that may affect operators during missions. We optimise the dataset, which includes the operator's physiological signals and inertial measurement unit (IMU) sensor data from the ROV, using data fusion approaches and feature extraction. We evaluate the dataset through the statistical analysis, illustrating a statistical difference between data characterising normal and involuntary commands. Finally, we evaluate the optimised and original datasets over a variety of classifiers, demonstrating an accuracy of 94% and 95%, respectively.

show abstract

“…Results showed that HR can be an appropriate measurement for identifying a stress condition, as the transition from analog to digital induced stress. Finally, a study published recently, proposes a newly designed wearable system for online cognitive workload monitoring [12]. Multiple bio-signals (RSP, HR, ST and pulse waveform) were monitored during high and low cognitive workload, also focusing on a novel energy-aware algorithm for low-power processing.…”

Section: A Joint Human -Uav Monitoringmentioning

confidence: 99%

An Analysis of Machine Learning Classifiers for Joint Operator-UAV Monitoring and Abnormal Command Detection

Elia

Savva

Pettemeridou

et al. 2022

2022 International Conference on Unmanned Aircraft Systems (ICUAS)

View full text Add to dashboard Cite

Unmanned aerial vehicles (UAVs) are widely used in search and rescue missions, which mostly involve stressful conditions for first responders and especially UAV operators, who have to control the UAVs under dynamically changing conditions and harsh environments. Operations under these conditions lead to increased stress and fatigue for the operators and may result in unwanted commands, which in turn may endanger the UAV's mission and result in accidents and delays, something often critical in search and rescue. However, jointly monitoring both the UAV and the operator's mental and physical status, we can potentially prevent such abnormal movements to be executed by the UAV. Wearable, non-invasive sensors and UAV's inertial measurement unit (IMU) sensor data, can be used in real-time to provide a holistic contextual awareness of the UAV and the operator. Through the use of machine learning and classification algorithms, we can identify such unwanted commands, enabling the UAV to ignore them. To this end, this paper presents an analysis and evaluation of the performance of a variety of machine learning classifiers using data from the IMU which is integrated in the UAV, and operator's biomarker data, collected and constructed in a realistic environment with a stress-induction technique. For each classifier, we analyze its performance in terms of accuracy, processing time and energy consumption, since the aim is to be deployed directly on the UAV. Our analysis suggests that a range of classifiers can be used in these situations and we also critically discuss trade-offs between accuracy, processing time and energy consumption.

show abstract

MBioTracker: Multimodal Self-Aware Bio-Monitoring Wearable System for Online Workload Detection

Cited by 19 publications

References 30 publications

Machine-Learning Based Monitoring of Cognitive Workload in Rescue Missions With Drones

Machine-Learning Based Monitoring of Cognitive Workload in Rescue Missions With Drones

A real‐world data collection framework for a fused dataset creation for joint human and remotely operated vehicle monitoring and anomalous command detection

An Analysis of Machine Learning Classifiers for Joint Operator-UAV Monitoring and Abnormal Command Detection

Contact Info

Product

Resources

About