BAUM-1: A Spontaneous Audio-Visual Face Database of Affective and Mental States

Zhalehpour, Sasan; Onder, Onur; Akhtar, Zahid; Erdem, Çiğdem Eroğlu

doi:10.1109/taffc.2016.2553038

Cited by 179 publications

(81 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Finally, the global feature is fed into a classifier to recognize facial expression. Extensive experiments on two public databases, i.e., the BAUM-1 [9], and eNTERFACE05 [10], show the effectiveness of our method.…”

Section: Related Workmentioning

confidence: 84%

“…To evaluate the recognition accuracy of our framework, extensive experiments on two public datasets of emotional videos were conducted: BAUM-1 [9] and eNTERFACE05 [10]. The implementation details are as follows.…”

Section: Experiments Studies and Results Analysismentioning

confidence: 99%

“…Specifically, in comparison with the previous best work, our method separately improves accuracy from 45.04% to 46.51% for the BAUM-1s database and from 42.16% to 43.72% on the eNTERFACE05 database. The methods in references [9,50] employ hand-crafted methods and shallow learning neural networks to extract low-level features. This also demonstrates that our high-level features perform better than hand-crafted features and indicates the validity of the deep learning method.…”

Section: Comparisons With State-of-the-art Methodsmentioning

confidence: 99%

“…The BAUM-1s database [9] has 1222 video samples from 31 people. They exhibited joy, anger, sadness, disgust, fear, surprise, boredom, and contempt.…”

Section: Datasetsmentioning

confidence: 99%

“…Convolutional neural networks (CNNs) have improved image classification accuracy and outperform the hand-crafted feature method owing to the large-scale datasets [8]. However, current emotional datasets are small and have noise [9,10]. Furthermore, spatial signals and temporal signals are very important for modeling emotional video.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Deep Temporal–Spatial Aggregation for Video-Based Facial Expression Recognition

Pan

Guo

et al. 2019

Symmetry

View full text Add to dashboard Cite

The proposed method has 30 streams, i.e., 15 spatial streams and 15 temporal streams. Each spatial stream corresponds to each temporal stream. Therefore, this work correlates with the symmetry concept. It is a difficult task to classify video-based facial expression owing to the gap between the visual descriptors and the emotions. In order to bridge the gap, a new video descriptor for facial expression recognition is presented to aggregate spatial and temporal convolutional features across the entire extent of a video. The designed framework integrates a state-of-the-art 30 stream and has a trainable spatial–temporal feature aggregation layer. This framework is end-to-end trainable for video-based facial expression recognition. Thus, this framework can effectively avoid overfitting to the limited emotional video datasets, and the trainable strategy can learn to better represent an entire video. The different schemas for pooling spatial–temporal features are investigated, and the spatial and temporal streams are best aggregated by utilizing the proposed method. The extensive experiments on two public databases, BAUM-1s and eNTERFACE05, show that this framework has promising performance and outperforms the state-of-the-art strategies.

show abstract

Section: Related Workmentioning

confidence: 84%

Section: Experiments Studies and Results Analysismentioning

confidence: 99%

Section: Comparisons With State-of-the-art Methodsmentioning

confidence: 99%

“…The BAUM-1s database [9] has 1222 video samples from 31 people. They exhibited joy, anger, sadness, disgust, fear, surprise, boredom, and contempt.…”

Section: Datasetsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Deep Temporal–Spatial Aggregation for Video-Based Facial Expression Recognition

Pan

Guo

et al. 2019

Symmetry

View full text Add to dashboard Cite

show abstract

Facial Expression Recognition in Real Time Using Convolutional Neural Network

Dhankar

Rathee

2021

Smart and Sustainable Intelligent Systems

View full text Add to dashboard Cite

Multimodal emotion recognition: A comprehensive review, trends, and challenges

Ramaswamy,

Palaniswamy

2024

WIREs Data Min & Knowl

View full text Add to dashboard Cite

Automatic emotion recognition is a burgeoning field of research and has its roots in psychology and cognitive science. This article comprehensively reviews multimodal emotion recognition, covering various aspects such as emotion theories, discrete and dimensional models, emotional response systems, datasets, and current trends. This article reviewed 179 multimodal emotion recognition literature papers from 2017 to 2023 to reflect on the current trends in multimodal affective computing. This article covers various modalities used in emotion recognition based on the emotional response system under four categories: subjective experience comprising text and self‐report; peripheral physiology comprising electrodermal, cardiovascular, facial muscle, and respiration activity; central physiology comprising EEG, neuroimaging, and EOG; behavior comprising facial, vocal, whole‐body behavior, and observer ratings. This review summarizes the measures and behavior of each modality under various emotional states. This article provides an extensive list of multimodal datasets and their unique characteristics. The recent advances in multimodal emotion recognition are grouped based on the research focus areas such as emotion elicitation strategy, data collection and handling, the impact of culture and modality on multimodal emotion recognition systems, feature extraction, feature selection, alignment of signals across the modalities, and fusion strategies. The recent multimodal fusion strategies are detailed in this article, as extracting shared representations of different modalities, removing redundant features from different modalities, and learning critical features from each modality are crucial for multimodal emotion recognition. This article summarizes the strengths and weaknesses of multimodal emotion recognition based on the review outcome, along with challenges and future work in multimodal emotion recognition. This article aims to serve as a lucid introduction, covering all aspects of multimodal emotion recognition for novices.This article is categorized under: Fundamental Concepts of Data and Knowledge > Human Centricity and User Interaction Technologies > Cognitive Computing Technologies > Artificial Intelligence

show abstract

BAUM-1: A Spontaneous Audio-Visual Face Database of Affective and Mental States

Cited by 179 publications

References 42 publications

Deep Temporal–Spatial Aggregation for Video-Based Facial Expression Recognition

Deep Temporal–Spatial Aggregation for Video-Based Facial Expression Recognition

Facial Expression Recognition in Real Time Using Convolutional Neural Network

Multimodal emotion recognition: A comprehensive review, trends, and challenges

Contact Info

Product

Resources

About