MetaAugment: Sample-Aware Data Augmentation Policy Learning

Abstract

Automated data augmentation has shown superior performance in image recognition. Existing works search for datasetlevel augmentation policies without considering individual sample variations, which are likely to be sub-optimal. On the other hand, learning different policies for different samples naively could greatly increase the computing cost. In this paper, we learn a sample-aware data augmentation policy efficiently by formulating it as a sample reweighting problem. Specifically, an augmentation policy network takes a transformation and the corresponding augmented image as inputs, and outputs a weight to adjust the augmented image loss computed by a task network. At training stage, the task network minimizes the weighted losses of augmented training images, while the policy network minimizes the loss of the task network on a validation set via meta-learning. We theoretically prove the convergence of the training procedure and further derive the exact convergence rate. Superior performance is achieved on widely-used benchmarks including CIFAR-10/100, Omniglot, and ImageNet.

An overview of mixing augmentation methods and augmentation strategies

“…This survey is focused on the two DA areas: data augmentation via mixing images and data augmentation policy selection (Cubuk et al 2019;Ho et al 2019;Lim et al 2019;Cubuk et al 2020;Hataya et al 2020;Li et al 2020;Zhang et al 2020;Zhou et al 2020;Mounsaveng et al 2020). The former genre is further divided into methods that erase part of the image Zhong et al 2020;Lopes et al 2019) and image mixing methods (Zhang et al 2018;Yun et al 2019;Guo et al 2019;Verma et al 2019;Lee et al 2020;Walawalkar et al 2020;Kim et al 2020;Inoue 2018;Tokozume et al 2018a;Summers and Dinneen 2019;Takahashi et al 2018;Hendrycks et al 2020;Zhou et al 2021;Huang et al 2020;Kim et al 2021;Uddin et al 2020) which can be further divided based on particular properties, e.g.…”

“…An open question, which we believe is worth investigation is application of DAPS methods to more complex DA techniques, going beyond traditional augmentations. The results presented in Figures 25, 26, 27 suggest that inclusion of more complex erasing or mixing (Cubuk et al 2019;Lim et al 2019;Ho et al 2019;Cubuk et al 2020;Hataya et al 2020;Li et al 2020;Zhang et al 2020;Zhou et al 2020;Mounsaveng et al 2020)…”

“…Another method in this group, MetaAugment (Zhou et al 2020), challenges one of the well established DA assumptions that an augmentation transformation is selected for the entire data set. MetaAugment evaluates a fit of the selected augmentation operation to the image and assigns a weight based on the significance of this fit.…”

“…Contrary to Adversarial AutoAugment (Zhang et al 2020) and MetaAugment (Zhou et al 2020) where subsequent training resulted in more adequate change of the method's parameters, in the case of ABO the transformations applied at the beginning of the training are stronger and approach identity towards the end of the training.…”

“…Another promising direction, which we believe will be explored in the near future, is image-based selection of DA techniques that aims at finding an optimal augmentation for each sample. One of possible realizations of this idea has been proposed in MetaAugment approach (Zhou et al 2020). There is also room for optimization based solutions that combine learning the optimal DA selection and parametrization with the target model training.…”