Analysing Affective Behavior in the First ABAW 2020 Competition

Kollias, Dimitrios; Schulc, Attila; Hajiyev, Elnar; Zafeiriou, Stefanos

doi:10.48550/arxiv.2001.11409

Cited by 16 publications

(29 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We validate our approach for emotion recognition on SEWA [34] and Aff-Wild2 [26,29,30,28,31,27,74]. We also validate our approach for the task of Action Unit intensity estimation on BP4D [75,65] and DISFA [42].…”

Section: Datasets and Performance Metricsmentioning

confidence: 89%

“…This is a longstanding problem in video recognition which has been extensively studied by the computer vision community [26,64,41,21,33,28,67,74,77,71,55,53,54,39,46]. Nevertheless, even recent methods [27,33,77] struggle to achieve high accuracy on the most difficult datasets including SEWA [34], Aff-Wild2 [29], BP4D [75] and DISFA [42]. Hence, there is still great progress to be made towards solving this challenging problem.…”

Section: Introductionmentioning

confidence: 99%

“…2 . AffWild2[27] is a large-scale video dataset divided in training, validation, and test partitions. Because the labels for the latter are not made publicly available, we use the training and validation only for our evaluation purposes.…”

mentioning

confidence: 99%

See 2 more Smart Citations

Affective Processes: stochastic modelling of temporal context for emotion and facial expression recognition

Sánchez¹,

Tellamekala²,

Valstar³

et al. 2021

Preprint

View full text Add to dashboard Cite

Temporal context is key to the recognition of expressions of emotion. Existing methods, that rely on recurrent or selfattention models to enforce temporal consistency, work on the feature level, ignoring the task-specific temporal dependencies, and fail to model context uncertainty. To alleviate these issues, we build upon the framework of Neural Processes to propose a method for apparent emotion recognition with three key novel components: (a) probabilistic contextual representation with a global latent variable model; (b) temporal context modelling using task-specific predictions in addition to features; and (c) smart temporal context selection. We validate our approach on four databases, two for Valence and Arousal estimation (SEWA and AffWild2), and two for Action Unit intensity estimation (DISFA and BP4D). Results show a consistent improvement over a series of strong baselines as well as over state-ofthe-art methods.

show abstract

Section: Datasets and Performance Metricsmentioning

confidence: 89%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Affective Processes: stochastic modelling of temporal context for emotion and facial expression recognition

Sánchez¹,

Tellamekala²,

Valstar³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Subsequently, to further progress the investigations in this field, the development of larger and bigger datasets was addressed. Several datasets have been introduced so far, starting with SEMAINE (McKeown et al, 2010), AFEW-VA (Kossaifi et al, 2017), RECOLA (Ringeval et al, 2013), OMG (Barros et al, 2018), AffectNet (Mollahosseini et al, 2015), and more recently SEWA (Kossaifi et al, 2019), aff-wild Zafeiriou et al, 2017), and aff-wild2 Kollias et al, 2020). Furthermore, the V-A labels have become the standard emotional dimensions over time, as opposed to hard emotion labels, given their continuous nature (Kossaifi et al, 2017;Kossaifi et al, 2019).…”

Section: Related Workmentioning

confidence: 99%

An Enhanced Adversarial Network with Combined Latent Features for Spatio-Temporal Facial Affect Estimation in the Wild

Aspandi,

Sukno,

Schuller

et al. 2021

Preprint

View full text Add to dashboard Cite

Affective Computing has recently attracted the attention of the research community, due to its numerous applications in diverse areas. In this context, the emergence of video-based data allows to enrich the widely used spatial features with the inclusion of temporal information. However, such spatio-temporal modelling often results in very high-dimensional feature spaces and large volumes of data, making training difficult and time consuming. This paper addresses these shortcomings by proposing a novel model that efficiently extracts both spatial and temporal features of the data by means of its enhanced temporal modelling based on latent features. Our proposed model consists of three major networks, coined Generator, Discriminator, and Combiner, which are trained in an adversarial setting combined with curriculum learning to enable our adaptive attention modules. In our experiments, we show the effectiveness of our approach by reporting our competitive results on both the AFEW-VA and SEWA datasets, suggesting that temporal modelling improves the affect estimates both in qualitative and quantitative terms. Furthermore, we find that the inclusion of attention mechanisms leads to the highest accuracy improvements, as its weights seem to correlate well with the appearance of facial movements, both in terms of temporal localisation and intensity. Finally, we observe the sequence length of around 160 ms to be the optimum one for temporal modelling, which is consistent with other relevant findings utilising similar lengths.

show abstract

“…In this paper, we summarize our approach and results for recognizing seven discrete emotions in the wild. The database is released as a part of the challenge and annotations are provided at frame level [3].…”

Section: Introductionmentioning

confidence: 99%

Expression Recognition in the Wild Using Sequence Modeling

Rasipuram,

Bhat,

Maitra

2020

Preprint

View full text Add to dashboard Cite

As we exceed upon the procedures for modelling the different aspects of behaviour, expression recognition has become a key field of research in Human Computer Interactions. Expression recognition in the wild is a very interesting problem and is challenging as it involves detailed feature extraction and heavy computation. This paper presents the methodologies and techniques used in our contribution to recognize different expressions i.e., neutral, anger, disgust, fear, happiness, sadness, surprise in ABAW competition on Aff-Wild2 database. Aff-Wild2 database consists of videos in the wild labelled for seven different expressions at frame level. We used a bi-modal approach by fusing audio and visual features and train a sequence-to-sequence model that is based on Gated Recurrent Units (GRU) and Long Short Term Memory (LSTM) network. We show experimental results on validation data. The overall accuracy of the proposed approach is 41.5 %, which is better than the competition baseline of 37%.

show abstract

Analysing Affective Behavior in the First ABAW 2020 Competition

Cited by 16 publications

References 28 publications

Affective Processes: stochastic modelling of temporal context for emotion and facial expression recognition

Affective Processes: stochastic modelling of temporal context for emotion and facial expression recognition

An Enhanced Adversarial Network with Combined Latent Features for Spatio-Temporal Facial Affect Estimation in the Wild

Expression Recognition in the Wild Using Sequence Modeling

Contact Info

Product

Resources

About