Emotion Classification with Data Augmentation Using Generative Adversarial Networks

Abstract

It is a difficult task to classify images with multiple class labels using only a small number of labeled examples, especially when the label (class) distribution is imbalanced. Emotion classification is such an example of imbalanced label distribution, because some classes of emotions like disgusted are relatively rare comparing to other labels like happy or sad. In this paper, we propose a data augmentation method using generative adversarial networks (GAN). It can complement and complete the data manifold and find better margins between neighboring classes. Specifically, we design a framework using a CNN model as the classifier and a cycle-consistent adversarial networks (CycleGAN) as the generator. In order to avoid gradient vanishing problem, we employ the least-squared loss as adversarial loss. We also propose several evaluation methods on three benchmark datasets to validate GAN's performance. Empirical results show that we can obtain 5%∼10% increase in the classification accuracy after employing the GAN-based data augmentation techniques.

Imbalance Problems in Object Detection: A Review

“…GANs) can also be used for extending the dataset to address imbalance. Many studies [141], [142], [143] have successfuly used GANs to generate examples for under-represented classes for various image classification problems.…”

Exploiting prior knowledge about biological macromolecules in cryo-EM structure determination

“…For this purpose, conditional GANs (cGAN) are a particularly suiting candidate, since pairs of ground truth and noisy data are available [39]. Similarly, cycle-consistent adversarial networks (CycleGAN) [40] or style transfer GANs [41] may be used to relate the two data domains in cases where pairs are lacking, e.g. for low resolution reconstructions without a matching ground truth or for generalizing the data set to new experimental conditions with few existing examples [42].…”

Online Hyper-Parameter Learning for Auto-Augmentation Strategy

“…There also exists attempts to learn data augmentations strategy, including Smart Augmentation [17], which proposed a network that automatically generates augmented data by merging two or more samples from the same class, and [29] which used a Bayesian approach to generate data based on the distribution learned from the training set. Generative adversarial networks have also been used for the purpose of generating additional data [22,35,1]. The work most closely related to our proposed method is [6], which formulated the auto-augmentation search as a discrete search problem and exploited a reinforcement learning framework to search the policy consisting of possible augmentation operations.…”