Heatmap-Guided Balanced Deep Convolution Networks for Family Classification in the Wild

Aspandi, Decky; Martínez, Oriol; Binefa, Xavier

doi:10.1109/fg.2019.8756557

Cited by 7 publications

(9 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The pipeline of our base model ANCLaF starts with the G network. It receives either the original input image I, or a distorted version of it, Ĩ, as detailed in (Aspandi et al, 2019c;Aspandi et al, 2019a). It simultaneously produces the cleaned reconstruction of the input image Î and a 2D latent representation that will be used as features (Z):…”

Section: Adversarial Network Withmentioning

confidence: 99%

See 1 more Smart Citation

An Enhanced Adversarial Network with Combined Latent Features for Spatio-Temporal Facial Affect Estimation in the Wild

Aspandi,

Sukno,

Schuller

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Affective Computing has recently attracted the attention of the research community, due to its numerous applications in diverse areas. In this context, the emergence of video-based data allows to enrich the widely used spatial features with the inclusion of temporal information. However, such spatio-temporal modelling often results in very high-dimensional feature spaces and large volumes of data, making training difficult and time consuming. This paper addresses these shortcomings by proposing a novel model that efficiently extracts both spatial and temporal features of the data by means of its enhanced temporal modelling based on latent features. Our proposed model consists of three major networks, coined Generator, Discriminator, and Combiner, which are trained in an adversarial setting combined with curriculum learning to enable our adaptive attention modules. In our experiments, we show the effectiveness of our approach by reporting our competitive results on both the AFEW-VA and SEWA datasets, suggesting that temporal modelling improves the affect estimates both in qualitative and quantitative terms. Furthermore, we find that the inclusion of attention mechanisms leads to the highest accuracy improvements, as its weights seem to correlate well with the appearance of facial movements, both in terms of temporal localisation and intensity. Finally, we observe the sequence length of around 160 ms to be the optimum one for temporal modelling, which is consistent with other relevant findings utilising similar lengths.

show abstract

Section: Adversarial Network Withmentioning

confidence: 99%

“…where f i is the total number of instances of discrete V-A classes i, and F is a normalisation factor (Aspandi et al, 2019a) for the total V-A classes (discretised by a value of 10). This normalisation factor is crucial in cases of large imbalance in the number of instances per class, like in the AFEW-VA dataset (see Section 4.1).…”

Section: Training Lossesmentioning

confidence: 99%

An Enhanced Adversarial Network with Combined Latent Features for Spatio-Temporal Facial Affect Estimation in the Wild

Aspandi,

Sukno,

Schuller

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…where n i is the total number of instances of discrete valence/arousal class i, and N is the normalisation factor [1] for the total valence/arousal class. This normalisation factor is crucial given considerably unbalanced class instance on the Aff-Wild2 dataset [17].…”

Section: B Conditional Discriminator-based Affect Estimatormentioning

confidence: 99%

Latent-Based Adversarial Neural Networks for Facial Affect Estimations

Aspandi

Mallol-Ragolta

Schuller

et al. 2020

2020 15th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2020)

Self Cite

View full text Add to dashboard Cite

There is a growing interest in affective computing research nowadays given its crucial role in bridging humans with computers. This progress has recently been accelerated due to the emergence of bigger dataset. One recent advance in this field is the use of adversarial learning to improve model learning through augmented samples. However, the use of latent features, which is feasible through adversarial learning, is not largely explored, yet. This technique may also improve the performance of affective models, as analogously demonstrated in related fields, such as computer vision. To expand this analysis, in this work, we explore the use of latent features through our proposed adversarial-based networks for valence and arousal recognition in the wild. Specifically, our models operate by aggregating several modalities to our discriminator, which is further conditioned to the extracted latent features by the generator. Our experiments on the recently released SEWA dataset suggest the progressive improvements of our results. Finally, we show our competitive results on the Affective Behavior Analysis in-the-Wild (ABAW) challenge dataset.

show abstract

Section: B Conditional Discriminator-based Affect Estimatormentioning

confidence: 99%

“…Our models were trained using an NVIDIA Titan X GPU and it took approximately two days to converge. The source code of our models is available at our github page 1…”

Section: E Model Trainingmentioning

confidence: 99%

Adversarial-based neural networks for affect estimations in the wild

Aspandi¹,

Mallol-Ragolta²,

Schuller³

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

There is a growing interest in affective computing research nowadays given its crucial role in bridging humans with computers. This progress has been recently accelerated due to the emergence of bigger data. One recent advance in this field is the use of adversarial learning to improve model learning through augmented samples. However, the use of latent features, which is feasible through adversarial learning, is not largely explored, yet. This technique may also improve the performance of affective models, as analogously demonstrated in related fields, such as computer vision. To expand this analysis, in this work, we explore the use of latent features through our proposed adversarial-based networks for valence and arousal recognition in the wild. Specifically, our models operate by aggregating several modalities to our discriminator, which is further conditioned to the extracted latent features by the generator. Our experiments on the recently released SEWA dataset suggest the progressive improvements of our results. Finally, we show our competitive results on the Affective Behavior Analysis in-the-Wild (ABAW) challenge dataset.

show abstract

Heatmap-Guided Balanced Deep Convolution Networks for Family Classification in the Wild

Cited by 7 publications

References 31 publications

An Enhanced Adversarial Network with Combined Latent Features for Spatio-Temporal Facial Affect Estimation in the Wild

An Enhanced Adversarial Network with Combined Latent Features for Spatio-Temporal Facial Affect Estimation in the Wild

Latent-Based Adversarial Neural Networks for Facial Affect Estimations

Adversarial-based neural networks for affect estimations in the wild

Contact Info

Product

Resources

About