ConvNets and ImageNet Beyond Accuracy: Understanding Mistakes and Uncovering Biases

Stock, Pierre; Cissé, Moustapha

doi:10.1007/978-3-030-01231-1_31

Cited by 122 publications

(100 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…This is in part due to the end-to-end nature of their training, which encourages models to exploit biased features if this leads to accurate classification. Prior works have mostly focused on uncovering and addressing different instances of bias in learned models, including gender bias [3,39,1] and racial bias [29]. However, the bias of the data itself has received less attention from the community.…”

Section: Related Workmentioning

confidence: 99%

REPAIR: Removing Representation Bias by Dataset Resampling

Vasconcelos

2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

210

126

View full text Add to dashboard Cite

Modern machine learning datasets can have biases for certain representations that are leveraged by algorithms to achieve high performance without learning to solve the underlying task. This problem is referred to as "representation bias". The question of how to reduce the representation biases of a dataset is investigated and a new dataset REPresentAtion bIas Removal (REPAIR) procedure is proposed. This formulates bias minimization as an optimization problem, seeking a weight distribution that penalizes examples easy for a classifier built on a given feature representation. Bias reduction is then equated to maximizing the ratio between the classification loss on the reweighted dataset and the uncertainty of the ground-truth class labels. This is a minimax problem that REPAIR solves by alternatingly updating classifier parameters and dataset resampling weights, using stochastic gradient descent. An experimental set-up is also introduced to measure the bias of any dataset for a given representation, and the impact of this bias on the performance of recognition models. Experiments with synthetic and action recognition data show that dataset REPAIR can significantly reduce representation bias, and lead to improved generalization of models trained on REPAIRed datasets. The tools used for characterizing representation bias, and the proposed dataset RE-PAIR algorithm, are available at https://github.com/ JerryYLi/Dataset-REPAIR/.

show abstract

Section: Related Workmentioning

confidence: 99%

REPAIR: Removing Representation Bias by Dataset Resampling

Vasconcelos

2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

210

126

View full text Add to dashboard Cite

show abstract

“…Recently, Stock and Cisse [49] presented empirical evidence that the performance of state-of-the-art classifiers on ImageNet [47] is largely underestimated -much of the reamining error is due to the fact that ImageNet's singlelabel annotation ignores the intrinsic multi-label nature of the images. Unlike ImageNet, multi-label datasets (e.g.…”

Section: Introductionmentioning

confidence: 99%

Learning a Deep ConvNet for Multi-Label Classification With Partial Labels

Durand

Mehrasa

Mori

2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

208

142

View full text Add to dashboard Cite

Deep ConvNets have shown great performance for single-label image classification (e.g. ImageNet), but it is necessary to move beyond the single-label classification task because pictures of everyday life are inherently multilabel. Multi-label classification is a more difficult task than single-label classification because both the input images and output label spaces are more complex. Furthermore, collecting clean multi-label annotations is more difficult to scale-up than single-label annotations. To reduce the annotation cost, we propose to train a model with partial labels i.e. only some labels are known per image. We first empirically compare different labeling strategies to show the potential for using partial labels on multi-label datasets. Then to learn with partial labels, we introduce a new classification loss that exploits the proportion of known labels per example. Our approach allows the use of the same training settings as when learning with all the annotations. We further explore several curriculum learning based strategies to predict missing labels. Experiments are performed on three large-scale multi-label datasets: MS COCO, NUS-WIDE and Open Images.

show abstract

“…In this paper, we have surfaced various oracle issues in ILSVRC2012. That errors and ambiguity exist in ImageNet data has been recognized for long [17,22,23], but as an academic benchmark focused on object classification, the presence of the right class in the top-5 has in the past years been seen as sufficient to consider the classification task solved. However, the oracles we currently have at hand, and the representation and evaluation frameworks we currently employ within ML, may be insufficient when considering the deployment of ML components in real-world application scenarios.…”

Section: Resultsmentioning

confidence: 99%

“…Beyond this, at a semantic and more conceptual level, human judgment of ground truth may have been partially erroneous, potentially inconsistent, or even ethically undesired to become part of an automated decision pipeline, e.g., because it may encode biased or offensive assessments [7,23,24,27]. Such issues raise questions on the true quality of the oracle, will have a serious influence on what an ML system will infer, and, from an experimental validity perspective, may cause systems to not actually learn what they are supposed to learn [24].…”

mentioning

confidence: 99%

Oracle Issues in Machine Learning and Where to Find Them

Liem

Panichella

2020

Proceedings of the IEEE/ACM 42nd International Conference on Software Engineering Workshops

View full text Add to dashboard Cite

The rise in popularity of machine learning (ML), and deep learning in particular, has both led to optimism about achievements of artificial intelligence, as well as concerns about possible weaknesses and vulnerabilities of ML pipelines. Within the software engineering community, this has led to a considerable body of work on ML testing techniques, including white-and black-box testing for ML models. This means the oracle problem needs to be addressed. For supervised ML applications, oracle information is indeed available in the form of dataset 'ground truth', that encodes input data with corresponding desired output labels. However, while ground truth forms a gold standard, there still is no guarantee it is truly correct. Indeed, syntactic, semantic, and conceptual framing issues in the oracle may negatively affect the ML system's integrity. While syntactic issues may automatically be verified and corrected, the higher-level issues traditionally need human judgment and manual analysis. In this paper, we employ two heuristics based on information entropy and semantic analysis on well-known computer vision models and benchmark data from ImageNet. The heuristics are used to semi-automatically uncover potential higher-level issues in (i) the label taxonomy used to define the ground truth oracle (labels), and (ii) data encoding and representation. In doing this, beyond existing ML testing efforts, we illustrate the need for software engineering strategies that especially target and assess the oracle.

show abstract

ConvNets and ImageNet Beyond Accuracy: Understanding Mistakes and Uncovering Biases

Cited by 122 publications

References 29 publications

REPAIR: Removing Representation Bias by Dataset Resampling

REPAIR: Removing Representation Bias by Dataset Resampling

Learning a Deep ConvNet for Multi-Label Classification With Partial Labels

Oracle Issues in Machine Learning and Where to Find Them

Contact Info

Product

Resources

About