Jointly Optimize Data Augmentation and Network Training: Adversarial Data Augmentation in Human Pose Estimation

Abstract

Random data augmentation is a critical technique to avoid overfitting in training deep neural network models. However, data augmentation and network training are usually treated as two isolated processes, limiting the effectiveness of network training. Why not jointly optimize the two? We propose adversarial data augmentation to address this limitation. The main idea is to design an augmentation network (generator) that competes against a target network (discriminator) by generating "hard" augmentation operations online. The augmentation network explores the weaknesses of the target network, while the latter learns from "hard" augmentations to achieve better performance. We also design a reward/penalty strategy for effective joint training. We demonstrate our approach on the problem of human pose estimation and carry out a comprehensive experimental analysis, showing that our method can significantly improve state-of-the-art models without additional data efforts.

Semantic Graph Convolutional Networks for 3D Human Pose Regression

“…These works focus on 2D to 3D pose regression, which are most relevant to the context of this paper. Other methods use synthetic datasets which are generated from deforming a human template model with the ground truth [8,42,48] or introduce loss functions involving high-level knowledge [40,53,69] in addition to joints. They are complementary to the others.…”

Predicting Animation Skeletons for 3D Articulated Models via Volumetric Nets

“…Our work is also related to 3D pose estimation methods that try to recover 3D locations of joints from 2D images or directly from 3D point cloud and volumetric data (see also [30,50] for related surveys). Most recent methods use deep architectures to extract joints for humans [48,19,41,33,38,75,61,42], hands [15,37,20,63,15,64,14], and more recently some species of animals [43]. However, all these approaches aim to predict a pre-defined set of joints for a particular class of objects.…”

Practical License Plate Recognition in Unconstrained Surveillance Systems with Adversarial Super-Resolution

“…e.g,. generative focused tasks; super-resolution (Nguyen et al, ;Ledig et al, 2017;Lee et al, 2018), style transfer (Zhu et al, 2017;Li et al, 2017), natural-language processing (Rajeswar et al, 2017) and discriminative focused tasks; human pose estimation (Chou et al, 2017;Peng et al, 2018).…”