Learning from binary labels with instance-dependent noise

Menon, Aditya Krishna; Rooyen, Brendan van; Natarajan, Nagarajan

doi:10.1007/s10994-018-5715-3

Cited by 67 publications

(77 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Improvements on this direction may also widen the applicability to massively multi-class scenarios. It remains an open question whether instance-dependent noise may be included into our approach [42,25]. Finally, we anticipate the use of our approach as a tool for pre-training models with noisy data from the Web, in the spirit of [17].…”

Section: Discussionmentioning

confidence: 99%

“…classdependent), label noise can produce solutions that are akin to random guessing [22]. On the other hand, the Bayes-optimal classifier remains unchanged under symmetric [28,26] and even instance dependent label noise [25] implying that highcapacity models are robust to essentially any level of such noise, given sufficiently many samples.…”

Section: Related Workmentioning

confidence: 99%

“…Classes may be too similar between each other for non-expert human labellers to distinguish, regardless of the specific instances. Little is known about learning under the more generic feature dependent noise, with few exceptions [42,8,25].…”

Section: Label Noise and Loss Robustnessmentioning

confidence: 99%

See 2 more Smart Citations

Making Deep Neural Networks Robust to Label Noise: A Loss Correction Approach

Patrini

Rozza²,

Menon

et al. 2017

2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

Self Cite

1,213

1,309

View full text Add to dashboard Cite

We present a theoretically grounded approach to train deep neural networks, including recurrent networks, subject to class-dependent label noise. We propose two procedures for loss correction that are agnostic to both application domain and network architecture. They simply amount to at most a matrix inversion and multiplication, provided that we know the probability of each class being corrupted into another. We further show how one can estimate these probabilities, adapting a recent technique for noise estimation to the multi-class setting, and thus providing an end-to-end framework. Extensive experiments on MNIST, IMDB, CIFAR-10, CIFAR-100 and a large scale dataset of clothing images employing a diversity of architectures -stacking dense, convolutional, pooling, dropout, batch normalization, word embedding, LSTM and residual layers -demonstrate the noise robustness of our proposals. Incidentally, we also prove that, when ReLU is the only non-linearity, the loss curvature is immune to class-dependent label noise.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Making Deep Neural Networks Robust to Label Noise: A Loss Correction Approach

Patrini

Rozza²,

Menon

et al. 2017

2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

Self Cite

1,213

1,309

View full text Add to dashboard Cite

show abstract

“…Segmentation with inaccurate or imprecise annotations refers to the scenario where the ground truth labels are corrupted with (random, class-conditional or instance-conditional [280], [281]) noises, thus also referring to noisy label learning [282], [283]. Imprecise boundaries, and mislabeling are also inaccurate annotations.…”

Section: Inaccurately-supervised Segmentationmentioning

confidence: 99%

Medical Image Segmentation With Limited Supervision: A Review of Deep Network Models

Wang

2021

IEEE Access

View full text Add to dashboard Cite

Despite the remarkable performance of deep learning methods on various tasks, most cutting-edge models rely heavily on large-scale annotated training examples, which are often unavailable for clinical and health care tasks. The labeling costs for medical images are very high, especially in medical image segmentation, which typically requires intensive pixel/voxel-wise labeling. Therefore, the strong capability of learning and generalizing from limited supervision, including a limited amount of annotations, sparse annotations, and inaccurate annotations, is crucial for the successful application of deep learning models in medical image segmentation. However, due to its intrinsic difficulty, segmentation with limited supervision is challenging and specific model design and/or learning strategies are needed. In this paper, we provide a systematic and up-to-date review of the solutions above, with summaries and comments about the methodologies. We also highlight several problems in this field, discussed future directions observing further investigations.

show abstract

“…They can either be learned in advance [12] or jointly with the rest of the model with an extra layer [13][14][15][16]. Prior work has also used a noise model conditioned on the input features [17,18]. However, these models cannot be directly applied to ASR as they do not handle sequential inputs and arbitrary-length outputs.…”

Section: Introductionmentioning

confidence: 99%

Lead2Gold: Towards Exploiting the Full Potential of Noisy Transcriptions for Speech Recognition

Dufraux

Hannun

Brun

et al. 2019

2019 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU)

View full text Add to dashboard Cite

The transcriptions used to train an Automatic Speech Recognition (ASR) system may contain errors. Usually, either a quality control stage discards transcriptions with too many errors, or the noisy transcriptions are used as is. We introduce Lead2Gold, a method to train an ASR system that exploits the full potential of noisy transcriptions. Based on a noise model of transcription errors, Lead2Gold searches for better transcriptions of the training data with a beam search that takes this noise model into account. The beam search is differentiable and does not require a forced alignment step, thus the whole system is trained end-to-end. Lead2Gold can be viewed as a new loss function that can be used on top of any sequence-to-sequence deep neural network. We conduct proof-of-concept experiments on noisy transcriptions generated from letter corruptions with different noise levels. We show that Lead2Gold obtains a better ASR accuracy than a competitive baseline which does not account for the (artificially-introduced) transcription noise.

show abstract

Learning from binary labels with instance-dependent noise

Cited by 67 publications

References 37 publications

Making Deep Neural Networks Robust to Label Noise: A Loss Correction Approach

Making Deep Neural Networks Robust to Label Noise: A Loss Correction Approach

Medical Image Segmentation With Limited Supervision: A Review of Deep Network Models

Lead2Gold: Towards Exploiting the Full Potential of Noisy Transcriptions for Speech Recognition

Contact Info

Product

Resources

About