Evading the Simplicity Bias: Training a Diverse Set of Models Discovers Solutions with Superior OOD Generalization

Teney, Damien; Abbasnejad, Ehsan; Lucey, Simon; Hengel, Anton van den

doi:10.48550/arxiv.2105.05612

Cited by 3 publications

(4 citation statements)

References 54 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Another group of papers proposes regularization techniques to learn diverse solutions on the train data, focusing on different groups of features [95,56,71,76]. Xu et al [101] show how to train orthogonal classifiers, i.e.…”

Section: Related Workmentioning

confidence: 99%

Last Layer Re-Training is Sufficient for Robustness to Spurious Correlations

Kirichenko¹,

Izmailov²,

Wilson³

2022

Preprint

View full text Add to dashboard Cite

Neural network classifiers can largely rely on simple spurious features, such as backgrounds, to make predictions. However, even in these cases, we show that they still often learn core features associated with the desired attributes of the data, contrary to recent findings. Inspired by this insight, we demonstrate that simple last layer retraining can match or outperform state-of-the-art approaches on spurious correlation benchmarks, but with profoundly lower complexity and computational expenses. Moreover, we show that last layer retraining on large ImageNet-trained models can also significantly reduce reliance on background and texture information, improving robustness to covariate shift, after only minutes of training on a single GPU.

show abstract

Section: Related Workmentioning

confidence: 99%

Last Layer Re-Training is Sufficient for Robustness to Spurious Correlations

Kirichenko¹,

Izmailov²,

Wilson³

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…In [20,19], a debiased dataset was generated using human labor. Various studies [2,33,47,57,62] have attempted to reduce dataset bias using explicit bias labels. These studies [2,33,47,57,45,44], used bias labels for each sample to reduce the influence of the bias labels when classifying target labels.…”

Section: Related Workmentioning

confidence: 99%

“…Furthermore, [61] proposed the EnD regularizer, which entangles target correlated features and disentangles biased attributes. Several studies [2,33,62] have designed DNNs as a shared feature extractors and multiple classifiers. In contrast to the shared feature extractor methods, [47,53] fabricated a classifier and conditional generative adversarial networks, yielding test samples to determine whether the classifier was biased.…”

Section: Related Workmentioning

confidence: 99%

Mitigating Dataset Bias by Using Per-sample Gradient

Ahn¹,

Kim²,

Yun³

2022

Preprint

View full text Add to dashboard Cite

The performance of deep neural networks is strongly influenced by the training dataset setup. In particular, when attributes having a strong correlation with the target attribute are present, the trained model can provide unintended prejudgments and show significant inference errors (i.e., the dataset bias problem). Various methods have been proposed to mitigate dataset bias, and their emphasis is on weakly correlated samples, called bias-conflicting samples. These methods are based on explicit bias labels involving human or empirical correlation metrics (e.g., training loss). However, such metrics require human costs or have insufficient theoretical explanation. In this study, we propose a debiasing algorithm, called PGD (Per-sample Gradient-based Debiasing), that comprises three steps: (1) training a model on uniform batch sampling, (2) setting the importance of each sample in proportion to the norm of the sample gradient, and (3) training the model using importance-batch sampling, whose probability is obtained in step (2). Compared with existing baselines for various synthetic and real-world datasets, the proposed method showed state-of-the-art accuracy for a the classification task. Furthermore, we describe theoretical understandings about how PGD can mitigate dataset bias. IntroductionDeep neural networks (DNNs) have been a research focus over the past decade owing to their humanlike performance in a range of tasks, including image classification [37], object detection [17], and image generation [18]. Despite their promising performance, the difficulties in obtaining wellcurated training datasets hinder the use of DNNs in real-world applications. In particular, dataset bias [63, 56], a bad training dataset problem, occurs when non-predictive attributes having a high correlation with the target attribute are present. This unintentionally correlated information (i.e., bias attributes) causes prejudice, and thereby leads to inaccurate inferences. For example, most "camel" images include a "desert background," and this unintended correlation can provide a false shortcut for answering "camel" on the basis of the the "desert." In [50,40], samples of data that have a strong correlation (like "desert background" in "camel" class images) are called "bias-aligned samples," while samples of data that have a weak correlation (like "camel on the grass" images) are termed "bias-conflicting samples."To reduce the dataset bias, initial studies [33,47,57,44] frequently assumed a case where labels with bias attributes are provided, but these additional labels provided through human effort are expensive. Alternatively, the bias-type, such as "background" ws assumed in [41,16,5,7,11]. However, assuming biased knowledge from humans is still unreasonable, since even humans cannot predict the type of bias that may exist in a large dataset [54]. Data for deep learning is typically collected by web-crawling without thorough consideration of the dataset bias problem.Recent studies [38,50,34,40] have replaced human intervention with D...

show abstract

“…EnD(Tartaglione, Barbano, and Grangetto 2021) proposes to entangle the target attribute and disengle the biased attributes. Multi-expert approaches(Alvi, Zisserman, and Nellåker 2018;Kim et al 2019;Teney et al 2021) use a shared feature extrator with multiple FC layers to classify multiple attributes independently. (McDuff et al 2019; Ramaswamy, Kim, and Russakovsky 2021) use conditional generator to determine if the trained classifier is biased.…”

mentioning

confidence: 99%

Denoising after Entropy-Based Debiasing a Robust Training Method for Dataset Bias with Noisy Labels

Ahn

Yun

2023

AAAI

View full text Add to dashboard Cite

Improperly constructed datasets can result in inaccurate inferences. For instance, models trained on biased datasets perform poorly in terms of generalization (i.e., dataset bias). Recent debiasing techniques have successfully achieved generalization performance by underestimating easy-to-learn samples (i.e., bias-aligned samples) and highlighting difficult-to-learn samples (i.e., bias-conflicting samples). However, these techniques may fail owing to noisy labels, because the trained model recognizes noisy labels as difficult-to-learn and thus highlights them. In this study, we find that earlier approaches that used the provided labels to quantify difficulty could be affected by the small proportion of noisy labels. Furthermore, we find that running denoising algorithms before debiasing is ineffective because denoising algorithms reduce the impact of difficult-to-learn samples, including valuable bias-conflicting samples. Therefore, we propose an approach called denoising after entropy-based debiasing, i.e., DENEB, which has three main stages. (1) The prejudice model is trained by emphasizing (bias-aligned, clean) samples, which are selected using a Gaussian Mixture Model. (2) Using the per-sample entropy from the output of the prejudice model, the sampling probability of each sample that is proportional to the entropy is computed. (3) The final model is trained using existing denoising algorithms with the mini-batches constructed by following the computed sampling probability. Compared to existing debiasing and denoising algorithms, our method achieves better debiasing performance on multiple benchmarks.

show abstract

Evading the Simplicity Bias: Training a Diverse Set of Models Discovers Solutions with Superior OOD Generalization

Cited by 3 publications

References 54 publications

Last Layer Re-Training is Sufficient for Robustness to Spurious Correlations

Last Layer Re-Training is Sufficient for Robustness to Spurious Correlations

Mitigating Dataset Bias by Using Per-sample Gradient

Denoising after Entropy-Based Debiasing a Robust Training Method for Dataset Bias with Noisy Labels

Contact Info

Product

Resources

About