Data augmentation instead of explicit regularization

Hernández-García, Álex; König, Peter

doi:10.48550/arxiv.1806.03852

Cited by 31 publications

(35 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Data Augmentation and Regularization Data augmentation expands the training data with examples generated via prior knowledge, which can be seen as an implicit regularization (Zhang et al, 2016;Hernández-García & König, 2018) where the prior is specified as virtual examples. Zhang et al (2018b) proposes mixup, which generate augmented samples via convex combinations of pairs of examples.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Fair Mixup: Fairness via Interpolation

Chuang,

Mroueh

2021

Preprint

View full text Add to dashboard Cite

Training classifiers under fairness constraints such as group fairness, regularizes the disparities of predictions between the groups. Nevertheless, even though the constraints are satisfied during training, they might not generalize at evaluation time. To improve the generalizability of fair classifiers, we propose fair mixup, a new data augmentation strategy for imposing the fairness constraint. In particular, we show that fairness can be achieved by regularizing the models on paths of interpolated samples between the groups. We use mixup, a powerful data augmentation strategy to generate these interpolates. We analyze fair mixup and empirically show that it ensures a better generalization for both accuracy and fairness measurement in tabular, vision, and language benchmarks. The code is available at https://github.com/chingyaoc/fair-mixup.

show abstract

Section: Related Workmentioning

confidence: 99%

“…On one hand, explicit regularization such as weight decay and dropout constrain the model capacity. On the other hand, implicit regularization such as data augmentation enlarge the support of the training distribution via prior knowledge (Hernández-García & König, 2018).…”

Section: Introductionmentioning

confidence: 99%

Fair Mixup: Fairness via Interpolation

Chuang,

Mroueh

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…We refer to these methodologies with the term data augmentation. Data augmentation approaches based on prior information (constraints) have been started to be explored in recent years, especially to cope with data sets of limited size and the related issue of poor generalization performance [12]. Data augmentation techniques have a strong history of success in the context of image-based learning tasks (e.g.…”

Section: Data Augmentationmentioning

confidence: 99%

Improving Deep Learning Models via Constraint-Based Domain Knowledge: a Brief Survey

Borghesi,

Baldo,

Milano

2020

Preprint

View full text Add to dashboard Cite

Deep Learning (DL) models proved themselves to perform extremely well on a wide variety of learning tasks, as they can learn useful patterns from large data sets. However, purely data-driven models might struggle when very difficult functions need to be learned or when there is not enough available training data. Fortunately, in many domains prior information can be retrieved and used to boost the performance of DL models.This paper presents a first survey of the approaches devised to integrate domain knowledge , expressed in the form of constraints, in DL learning models to improve their performance, in particular targeting deep neural networks. We identify five (non-mutually exclusive) categories that encompass the main approaches to inject domain knowledge: 1) acting on the features space, 2) modifications to the hypothesis space, 3) data augmentation, 4) regularization schemes, 5) constrained learning.

show abstract

“…Data augmentation has significantly improved the generalization of deep neural networks on a variety of image tasks, including image classification (Perez & Wang, 2017;De-Vries & Taylor, 2017), object detection (Zhong et al, 2017;Guo & Gould, 2015), and instance segmentation (Wang et al, 2018). Prior work has shown that data augmentation on its own can perform better than, or on par with, highly regularized models using other regularization techniques such as dropout (Hernández-García & König, 2018). This effectiveness is especially prominent in low data regimes, where models often fail to capture the full variance of the data in the training set (Zhang et al, 2019).…”

Section: Introductionmentioning

confidence: 99%

Data augmentation with Mobius transformations

Zhou

Zhang

Jiang

et al. 2021

Mach. Learn.: Sci. Technol.

View full text Add to dashboard Cite

Data augmentation has led to substantial improvements in the performance and generalization of deep models, and remains a highly adaptable method to evolving model architectures and varying amounts of data—in particular, extremely scarce amounts of available training data. In this paper, we present a novel method of applying Möbius transformations to augment input images during training. Möbius transformations are bijective conformal maps that generalize image translation to operate over complex inversion in pixel space. As a result, Möbius transformations can operate on the sample level and preserve data labels. We show that the inclusion of Möbius transformations during training enables improved generalization over prior sample-level data augmentation techniques such as cutout and standard crop-and-flip transformations, most notably in low data regimes.

show abstract

Data augmentation instead of explicit regularization

Cited by 31 publications

References 0 publications

Fair Mixup: Fairness via Interpolation

Fair Mixup: Fairness via Interpolation

Improving Deep Learning Models via Constraint-Based Domain Knowledge: a Brief Survey

Data augmentation with Mobius transformations

Contact Info

Product

Resources

About