Multi-EmoNet: A Novel Multi-Task Neural Network for Driver Emotion Recognition

Cui, Yaodong; Ma, Yintao; Li, Wenbo; Bian, Ning; Li, Guofa; Cao, Dongpu

doi:10.1016/j.ifacol.2021.04.155

Cited by 13 publications

(2 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As shown in Table 5, most of the facialexpression-based studies choose the discrete emotion (six basic emotions and neutral) as the detection targets, and the deep neural network, such as CNN-based models, are commonly used in facial classification, such as GLFCNN [82], Xception [86,87], and basic CNN [85,90]. Equations ( 1) and (2) show the most commonly used loss function (multiclass cross-entropy loss and binary cross-entropy loss) for facial-expression-based emotion detection [82,85,86,88].…”

Section: Facial-expression-basedmentioning

confidence: 99%

Review and Perspectives on Human Emotion for Connected Automated Vehicles

Li,

Tan

et al. 2024

Automot. Innov.

Self Cite

View full text Add to dashboard Cite

The progression toward automated driving and the latest advancement in vehicular networking have led to novel and natural human-vehicle-road systems, in which affective human-vehicle interaction is a crucial factor affecting the acceptance, safety, comfort, and traffic efficiency of connected and automated vehicles (CAVs). This development has inspired increasing interest in how to develop affective interaction framework for intelligent cockpit in CAVs. To enable affective human-vehicle interactions in CAVs, knowledge from multiple research areas is needed, including automotive engineering, transportation engineering, human–machine interaction, computer science, communication, as well as industrial engineering. However, there is currently no systematic survey considering the close relationship between human-vehicle-road and human emotion in the human-vehicle-road coupling process in the CAV context. To facilitate progress in this area, this paper provides a comprehensive literature survey on emotion-related studies from multi-aspects for better design of affective interaction in intelligent cockpit for CAVs. This paper discusses the multimodal expression of human emotions, investigates the human emotion experiment in driving, and particularly emphasizes previous knowledge on human emotion detection, regulation, as well as their applications in CAVs. The promising research perspectives are outlined for researchers and engineers from different research areas to develop CAVs with better acceptance, safety, comfort, and enjoyment for users.

show abstract

Section: Facial-expression-basedmentioning

confidence: 99%

Review and Perspectives on Human Emotion for Connected Automated Vehicles

Li,

Tan

et al. 2024

Automot. Innov.

Self Cite

View full text Add to dashboard Cite

show abstract

“…Emotion recognition is an important part of the interaction between people and machines [1], it has broad application prospects in the fields of distance education [2], psychological therapy [3], assisted driving [4], etc. Now considerable work about emotion recognition has been performed, but there still exists some obstacles toward accurate emotion recognition.…”

Section: Introductionmentioning

confidence: 99%

A novel dual-modal emotion recognition algorithm with fusing hybrid features of audio signal and speech context

et al. 2022

Complex Intell. Syst.

View full text Add to dashboard Cite

With regard to human–machine interaction, accurate emotion recognition is a challenging problem. In this paper, efforts were taken to explore the possibility to complete the feature abstraction and fusion by the homogeneous network component, and propose a dual-modal emotion recognition framework that is composed of a parallel convolution (Pconv) module and attention-based bidirectional long short-term memory (BLSTM) module. The Pconv module employs parallel methods to extract multidimensional social features and provides more effective representation capacity. Attention-based BLSTM module is utilized to strengthen key information extraction and maintain the relevance between information. Experiments conducted on the CH-SIMS dataset indicate that the recognition accuracy reaches 74.70% on audio data and 77.13% on text, while the accuracy of the dual-modal fusion model reaches 90.02%. Through experiments it proves the feasibility to process heterogeneous information within homogeneous network component, and demonstrates that attention-based BLSTM module would achieve best coordination with the feature fusion realized by Pconv module. This can give great flexibility for the modality expansion and architecture design.

show abstract