Recycling: Semi-Supervised Learning With Noisy Labels in Deep Neural Networks

Kong, Kyeongbo; Lee, Junggi; Kwak, Youngchul; Kang, Min-Sung; Kim, Seong Gyun; Song, Woo‐Jin

doi:10.1109/access.2019.2918794

Cited by 33 publications

(24 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…4b), and mixed flipping (Fig. 4c) (Tanaka et al, 2018;Han et al, 2018c;Song et al, 2019;Yu et al, 2019;Kong et al, 2019).…”

Section: B2 Datasetmentioning

confidence: 96%

“…We used CIFAR-10 ( Krizhevsky et al, 2009), CIFAR-100 (Krizhevsky et al, 2009), and Tiny-ImageNet (Tin), which are widely used benchmark datasets in the noisy label fields (Han et al, 2018c;Song et al, 2019;Yu et al, 2019;Kong et al, 2019;Arazo et al, 2019;Chen et al, 2019;Kim et al, 2019). CIFAR-10 and CIFAR-100 consist of 10 and 100 classes, respectively, of 32 × 32 color images.…”

Section: B2 Datasetmentioning

confidence: 99%

“…Recently, to overcome this limitation, many studies have been based on the sample selection method. In the sample selection category, only clean samples, whose labels are accurate, are selected from the mini-batch and thereafter used to train the network (Han et al, 2018c;Malach & Shalev-Shwartz, 2017;Wang et al, 2019a;Yu et al, 2019;Shen & Sanghavi, 2019;Song et al, 2019;Kong et al, 2019)'. They are based on "content-aware optimization" (also called "memorization effect"), wherein it initially learns simple patterns shared by multiple training samples across the data regardless of their labels (Arpit et al, 2017).…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Mitigating Memorization in Sample Selection for Learning with Noisy Labels

Kong,

Lee,

Kwak

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Because deep learning is vulnerable to noisy labels, sample selection techniques, which train networks with only clean labeled data, have attracted a great attention. However, if the labels are dominantly corrupted by few classes, these noisy samples are called dominant-noisy-labeled samples, the network also learns dominant-noisy-labeled samples rapidly via content-aware optimization. In this study, we propose a compelling criteria to penalize dominant-noisy-labeled samples intensively through class-wise penalty labels. By averaging prediction confidences for the each observed label, we obtain suitable penalty labels that have high values if the labels are largely corrupted by some classes. Experiments were performed using benchmarks (CIFAR-10, CIFAR-100, Tiny-ImageNet) and real-world datasets (ANIMAL-10N, Clothing1M) to evaluate the proposed criteria in various scenarios with different noise rates. Using the proposed sample selection, the learning process of the network becomes significantly robust to noisy labels compared to existing methods in several noise types.

show abstract

“…4b), and mixed flipping (Fig. 4c) (Tanaka et al, 2018;Han et al, 2018c;Song et al, 2019;Yu et al, 2019;Kong et al, 2019).…”

Section: B2 Datasetmentioning

confidence: 96%

Section: B2 Datasetmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Mitigating Memorization in Sample Selection for Learning with Noisy Labels

Kong,

Lee,

Kwak

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Based on this, Recycling, 14 which belongs to the second approach mentioned in Section 1, improves the generalization of the network through semisupervised learning. It first selects clean and noisy samples using the sample selection method based on small‐loss criteria that assumes that the probability of a sample being clean is high as the loss of the sample is low.…”

Section: Background Knowledgementioning

confidence: 99%

“…In the first approach, 12,13 clean samples are selected among the training data, and the network is trained using only the selected samples. In the second approach, 14,15 clean samples are separated from noisy samples, and the network is trained using clean samples with original labels and noisy samples with corrected labels. In general, a better performance is achieved with the second approach than with the first because more data are used for training.…”

Section: Introductionmentioning

confidence: 99%