Driver Facial Landmark Detection in Real Driving Situations

Jeong, Mira; Ko, Byoung Chul; Kwak, Sooyeong; Nam, Jae-Yeal

doi:10.1109/tcsvt.2017.2769096

Cited by 25 publications

(12 citation statements)

References 44 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…FL models are trained model from the appearance and shape variations from a coarse initialization. Then, the initial shape is moved to a better position step-by-step until convergence [ 16 ]. Basic emotions (BEs) are seven basic human emotions: happiness, surprise, anger, sadness, fear, disgust, and neutral, as shown in Figure 3 a.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A Brief Review of Facial Emotion Recognition Based on Visual Information

2018

Sensors

536

239

View full text Add to dashboard Cite

Facial emotion recognition (FER) is an important topic in the fields of computer vision and artificial intelligence owing to its significant academic and commercial potential. Although FER can be conducted using multiple sensors, this review focuses on studies that exclusively use facial images, because visual expressions are one of the main information channels in interpersonal communication. This paper provides a brief review of researches in the field of FER conducted over the past decades. First, conventional FER approaches are described along with a summary of the representative categories of FER systems and their main algorithms. Deep-learning-based FER approaches using deep networks enabling “end-to-end” learning are then presented. This review also focuses on an up-to-date hybrid deep-learning approach combining a convolutional neural network (CNN) for the spatial features of an individual frame and long short-term memory (LSTM) for temporal features of consecutive frames. In the later part of this paper, a brief review of publicly available evaluation metrics is given, and a comparison with benchmark results, which are a standard for a quantitative comparison of FER researches, is described. This review can serve as a brief guidebook to newcomers in the field of FER, providing basic knowledge and a general understanding of the latest state-of-the-art studies, as well as to experienced researchers looking for productive directions for future work.

show abstract

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

A Brief Review of Facial Emotion Recognition Based on Visual Information

2018

Sensors

536

239

View full text Add to dashboard Cite

show abstract

“…The geometric features generally include the relationship between facial components. Facial landmark points are representative examples of geometric features [2,30,31]. The global facial region features or different types of information on facial regions are extracted as appearance features [20,36].…”

Section: Facial Emotion Recognitionmentioning

confidence: 99%

“…The most important issue in the speech-emotion recognition system is the effective parallel use of the extraction of proper speech-signal features and an appropriate classification engine. These features include pitch, formant, and energy features [23,33,41]. In addition, the mel-frequency cepstrum coefficients (MFCC) feature is representatively used in many studies for speech-emotion recognition [26,37,39].…”

Section: Introductionmentioning

confidence: 99%

Human emotion recognition based on the weighted integration method using image sequences and acoustic features

Byun

Lee

2020

Multimed Tools Appl

View full text Add to dashboard Cite

People generally perceive other people’s emotions based on speech and facial expressions, so it can be helpful to use speech signals and facial images simultaneously. However, because the characteristics of speech and image data are different, combining the two inputs is still a challenging issue in the area of emotion-recognition research. In this paper, we propose a method to recognize emotions by synchronizing speech signals and image sequences. We design three deep networks. One of the networks is trained using image sequences, which focus on facial expression changes. Facial landmarks are also input to another network to reflect facial motion. The speech signals are first converted to acoustic features, which are used for the input of the other network, synchronizing the image sequence. These three networks are combined using a novel integration method to boost the performance of emotion recognition. A test comparing accuracy is conducted to verify the proposed method. The results demonstrated that the proposed method exhibits more accurate performance than previous studies.

show abstract

“…However, if a decision tree is not removed, the weight of each tree is generated according to its accuracy, estimated based on OOB. Because OOB data are not involved in the building of the tree, the weight learning from this dataset can avoid over-fitting [45]. We can repeat the above procedure to generate the T′ decision trees and the accuracy values of the remaining trees are utilized as the weights wt.…”

Section: Facial Expression Recognition Approachmentioning

confidence: 99%

Driver’s Facial Expression Recognition in Real-Time for Safe Driving

Jeong

2018

Sensors

Self Cite

136

View full text Add to dashboard Cite

In recent years, researchers of deep neural networks (DNNs)-based facial expression recognition (FER) have reported results showing that these approaches overcome the limitations of conventional machine learning-based FER approaches. However, as DNN-based FER approaches require an excessive amount of memory and incur high processing costs, their application in various fields is very limited and depends on the hardware specifications. In this paper, we propose a fast FER algorithm for monitoring a driver’s emotions that is capable of operating in low specification devices installed in vehicles. For this purpose, a hierarchical weighted random forest (WRF) classifier that is trained based on the similarity of sample data, in order to improve its accuracy, is employed. In the first step, facial landmarks are detected from input images and geometric features are extracted, considering the spatial position between landmarks. These feature vectors are then implemented in the proposed hierarchical WRF classifier to classify facial expressions. Our method was evaluated experimentally using three databases, extended Cohn-Kanade database (CK+), MMI and the Keimyung University Facial Expression of Drivers (KMU-FED) database, and its performance was compared with that of state-of-the-art methods. The results show that our proposed method yields a performance similar to that of deep learning FER methods as 92.6% for CK+ and 76.7% for MMI, with a significantly reduced processing cost approximately 3731 times less than that of the DNN method. These results confirm that the proposed method is optimized for real-time embedded applications having limited computing resources.

show abstract

Driver Facial Landmark Detection in Real Driving Situations

Cited by 25 publications

References 44 publications

A Brief Review of Facial Emotion Recognition Based on Visual Information

A Brief Review of Facial Emotion Recognition Based on Visual Information

Human emotion recognition based on the weighted integration method using image sequences and acoustic features

Driver’s Facial Expression Recognition in Real-Time for Safe Driving

Contact Info

Product

Resources

About