Driver Attention Assessment Using Physiological Measures from EEG, ECG, and EDA Signals

Individuals that pay attention to narrative stimuli show synchronized heart rate (HR) and electrodermal activity (EDA) responses. The degree to which this physiological synchrony occurs is related to attentional engagement. Factors that can influence attention, such as instructions, salience of the narrative stimulus and characteristics of the individual, affect physiological synchrony. The demonstrability of synchrony depends on the amount of data used in the analysis. We investigated how demonstrability of physiological synchrony varies with varying group size and stimulus duration. Thirty participants watched six 10 min movie clips while their HR and EDA were monitored using wearable sensors (Movisens EdaMove 4 and Wahoo Tickr, respectively). We calculated inter-subject correlations as a measure of synchrony. Group size and stimulus duration were varied by using data from subsets of the participants and movie clips in the analysis. We found that for HR, higher synchrony correlated significantly with the number of answers correct for questions about the movie, confirming that physiological synchrony is associated with attention. For both HR and EDA, with increasing amounts of data used, the percentage of participants with significant synchrony increased. Importantly, we found that it did not matter how the amount of data was increased. Increasing the group size or increasing the stimulus duration led to the same results. Initial comparisons with results from other studies suggest that our results do not only apply to our specific set of stimuli and participants. All in all, the current work can act as a guideline for future research, indicating the amount of data minimally needed for robust analysis of synchrony based on inter-subject correlations.

Section: Introductionmentioning

confidence: 99%

Robustness of Physiological Synchrony in Wearable Electrodermal Activity and Heart Rate as a Measure of Attentional Engagement to Movie Clips

Stuldreher

Erp

Brouwer

2023

“…The experiment was designed to induce high and low mental engagement in drivers through manual and autonomous driving tasks, respectively. This approach was chosen based on our previous research findings, which showed that manual driving elicits higher mental engagement compared to autonomous driving, and these differences can be detected through EEG, SPR, and ECG signals [36,37]. Accordingly, to represent high engagement, a label of "0" was assigned to all manual driving segments, while a label of "1" was assigned to all autonomous driving segments, representing low engagement.…”

Section: Signal Segmentationmentioning

confidence: 99%

Drivers’ Mental Engagement Analysis Using Multi-Sensor Fusion Approaches Based on Deep Convolutional Neural Networks

Aminosharieh Najafi,

Affanni,

Rinaldo

et al. 2023

Self Cite

In this paper, we present a comprehensive assessment of individuals’ mental engagement states during manual and autonomous driving scenarios using a driving simulator. Our study employed two sensor fusion approaches, combining the data and features of multimodal signals. Participants in our experiment were equipped with Electroencephalogram (EEG), Skin Potential Response (SPR), and Electrocardiogram (ECG) sensors, allowing us to collect their corresponding physiological signals. To facilitate the real-time recording and synchronization of these signals, we developed a custom-designed Graphical User Interface (GUI). The recorded signals were pre-processed to eliminate noise and artifacts. Subsequently, the cleaned data were segmented into 3 s windows and labeled according to the drivers’ high or low mental engagement states during manual and autonomous driving. To implement sensor fusion approaches, we utilized two different architectures based on deep Convolutional Neural Networks (ConvNets), specifically utilizing the Braindecode Deep4 ConvNet model. The first architecture consisted of four convolutional layers followed by a dense layer. This model processed the synchronized experimental data as a 2D array input. We also proposed a novel second architecture comprising three branches of the same ConvNet model, each with four convolutional layers, followed by a concatenation layer for integrating the ConvNet branches, and finally, two dense layers. This model received the experimental data from each sensor as a separate 2D array input for each ConvNet branch. Both architectures were evaluated using a Leave-One-Subject-Out (LOSO) cross-validation approach. For both cases, we compared the results obtained when using only EEG signals with the results obtained by adding SPR and ECG signals. In particular, the second fusion approach, using all sensor signals, achieved the highest accuracy score, reaching 82.0%. This outcome demonstrates that our proposed architecture, particularly when integrating EEG, SPR, and ECG signals at the feature level, can effectively discern the mental engagement of drivers.

“…The power spectral density was calculated by the Lomb-Scargle periodogram [60]. Based on original research articles [31,[35][36][37]42,47,[61][62][63], 14 HRV features listed in Table 1 that might be the potential to differentiate concentration and distraction were computed.…”

Section: Physiological Signal Preprocessing and Feature Computationmentioning

confidence: 99%

Monitoring Inattention in Construction Workers Caused by Physical Fatigue Using Electrocardiograph (ECG) and Galvanic Skin Response (GSR) Sensors

Ouyang,

Liu,

Cheng

et al. 2023

Physical fatigue is frequent for heavy manual laborers like construction workers, but it causes distraction and may lead to safety incidents. The purpose of this study is to develop predictive models for monitoring construction workers’ inattention caused by physical fatigue utilizing electrocardiograph (ECG) and galvanic skin response (GSR) sensors. Thirty participants were invited to complete an attention-demanding task under non-fatigued and physically fatigued conditions. Supervised learning algorithms were utilized to develop models predicting their attentional states, with heart rate variability (HRV) features derived from ECG signals and skin electric activity features derived from GSR signals as data inputs. The results demonstrate that using HRV features alone could obtain a prediction accuracy of 88.33%, and using GSR features alone could achieve an accuracy of 76.67%, both through the KNN algorithm. The accuracy increased to 96.67% through the SVM algorithm when combining HRV and GSR features. The findings indicate that ECG sensors used alone or in combination with GSR sensors can be applied to monitor construction workers’ inattention on job sites. The findings would provide an approach for detecting distracted workers at job sites. Additionally, it might reveal the relationships between workers’ physiological features and attention.