A Spontaneous Driver Emotion Facial Expression (DEFE) Dataset for Intelligent Vehicles: Emotions Triggered by Video-Audio Clips in Driving Scenarios

Li, Wenbo; Cui, Yaodong; Ma, Yintao; Chen, Xingxin; Li, Guofa; Zeng, Guanzhong; Guo, Gang; Cao, Dongpu

doi:10.1109/taffc.2021.3063387

Cited by 47 publications

(26 citation statements)

References 54 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, the MDM dataset focuses on the drivers, not the road scenes. While there has been interest in different aspects of driver monitoring such as visual attention [23], emotions [24], and driving anomalies [25], [26], we will review relevant datasets that have focused on head pose estimation (Sec. II-A), and gaze estimation (Sec.…”

Section: Related Workmentioning

confidence: 99%

The Multimodal Driver Monitoring Database: A Naturalistic Corpus to Study Driver Attention

Jha

Marzban

et al. 2022

IEEE Trans. Intell. Transport. Syst.

View full text Add to dashboard Cite

A smart vehicle should be able to monitor the actions and behaviors of the human driver to provide critical warnings or intervene when necessary. Recent advancements in deep learning and computer vision have shown great promise in monitoring human behavior and activities. While these algorithms work well in a controlled environment, naturalistic driving conditions add new challenges such as illumination variations, occlusions, and extreme head poses. A vast amount of in-domain data is required to train models that provide high performance in predicting driving related tasks to effectively monitor driver actions and behaviors. Toward building the required infrastructure, this paper presents the multimodal driver monitoring (MDM) dataset, which was collected with 59 subjects that were recorded performing various tasks. We use the Fi-Cap device that continuously tracks the head movement of the driver using fiducial markers, providing frame-based annotations to train head pose algorithms in naturalistic driving conditions. We ask the driver to look at predetermined gaze locations to obtain accurate correlation between the driver's facial image and visual attention. We also collect data when the driver performs common secondary activities such as navigation using a smart phone and operating the in-car infotainment system. All of the driver's activities are recorded with high definition RGB cameras and a time-of-flight depth camera. We also record the controller area network-bus (CAN-Bus), extracting important information. These high quality recordings serve as the ideal resource to train various efficient algorithms for monitoring the driver, providing further advancements in the field of in-vehicle safety systems.

show abstract

Section: Related Workmentioning

confidence: 99%

The Multimodal Driver Monitoring Database: A Naturalistic Corpus to Study Driver Attention

Jha

Marzban

et al. 2022

IEEE Trans. Intell. Transport. Syst.

View full text Add to dashboard Cite

show abstract

“…The development of the smart cockpit will expand and deepen the scope of HMI between humans and vehicles, resulting in new human-vehicle interaction problems that challenge safety, comfort, and driver's acceptance [4]. Among the problems, emotion-aware human-vehicle interactions are urgently needed to be addressed for improvement [5].…”

Section: Introductionmentioning

confidence: 99%

“…Drivers' emotions can be recognized and regulated in various ways by the emotion-aware HMI of the smart cockpit, based on which the safety and comfort of driving can be improved. As the first step for the development of the emotion-aware HMI system, precise driver emotion recognition is of great significance to realize the above improvement [4].…”

Section: Introductionmentioning

confidence: 99%

“…The emotions of humans can be expressed in forms including behavioral expressions and physiological changes. Up to present, various behavioral measurements [8], [9], physiological signal measurements [10], [11], or self-reported scales [4], [12] have been applied to driver emotions recognition. Considering the significant impact of the interference and intrusion on the emotional expression and real emotion experience, the application of non-invasive, non-contact, continuous measurement methods during the study of driver emotions is essential [4], [13].…”

Section: Introductionmentioning

confidence: 99%

“…Up to present, various behavioral measurements [8], [9], physiological signal measurements [10], [11], or self-reported scales [4], [12] have been applied to driver emotions recognition. Considering the significant impact of the interference and intrusion on the emotional expression and real emotion experience, the application of non-invasive, non-contact, continuous measurement methods during the study of driver emotions is essential [4], [13]. Therefore, in this study, the facial expressions of the driver are adopted as the main information to recognize the drivers emotion.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

CogEmoNet: A Cognitive-Feature-Augmented Driver Emotion Recognition Model for Smart Cockpit

Zeng

Zhang

et al. 2022

IEEE Trans. Comput. Soc. Syst.

Self Cite

View full text Add to dashboard Cite

Driver's emotion recognition is vital to improving driving safety, comfort, and acceptance of intelligent vehicles. This paper presents a cognitive-feature-augmented driver emotion detection method which is based on emotional cognitive process theory and deep networks. Different from the traditional methods, both the driver's facial expression and cognitive process characteristics (age, gender, and driving age) were used as the inputs of the proposed model. Convolutional techniques were adopted to construct the model for drivers emotion detection simultaneously considering the drivers facial expression and cognitive process characteristics. A driver's emotion data collection was carried out to validate the performance of the proposed method. The collected dataset consists of 40 drivers' frontal facial videos, their cognitive process characteristics and self-reported assessments on driver emotions. Another two deep networks were also used to compare recognition performance. The results prove that proposed method can achieve well detection results for different databases on the discrete emotion model and dimensional emotion model, respectively.

show abstract

Ergonomics Evaluation of In-Vehicle HMI Based on Meander of Finger Trajectory

Tang

Zhang

Guo

2022

HCI in Mobility, Transport, and Automotive Systems

View full text Add to dashboard Cite

A Spontaneous Driver Emotion Facial Expression (DEFE) Dataset for Intelligent Vehicles: Emotions Triggered by Video-Audio Clips in Driving Scenarios

Cited by 47 publications

References 54 publications

The Multimodal Driver Monitoring Database: A Naturalistic Corpus to Study Driver Attention

The Multimodal Driver Monitoring Database: A Naturalistic Corpus to Study Driver Attention

CogEmoNet: A Cognitive-Feature-Augmented Driver Emotion Recognition Model for Smart Cockpit

Ergonomics Evaluation of In-Vehicle HMI Based on Meander of Finger Trajectory

Contact Info

Product

Resources

About