Tripartite: Tackle Noisy Labels by a More Precise Partition

Abstract

Samples in large-scale datasets may be mislabeled due to various reasons, and Deep Neural Networks can easily over-fit to the noisy label data. To tackle this problem, the key point is to alleviate the harm of these noisy labels. Many existing methods try to divide training data into clean and noisy subsets in terms of loss values, and then process the noisy label data variedly. One of the reasons hindering a better performance is the hard samples. As hard samples always have relatively large losses whether their labels are clean or noisy, these methods could not divide them precisely. Instead, we propose a Tripartite solution to partition training data more precisely into three subsets: hard, noisy, and clean. The partition criteria are based on the inconsistent predictions of two networks, and the inconsistency between the prediction of a network and the given label. To minimize the harm of noisy labels but maximize the value of noisy label data, we apply a low-weight learning on hard data and a self-supervised learning on noisy label data without using the given labels. Extensive experiments demonstrate that Tripartite can filter out noisy label data more precisely, and outperforms most state-of-the-art methods on five benchmark datasets, especially on real-world datasets.

Contrastive Learning Joint Regularization for Pathological Image Classification with Noisy Labels

“…UNICOM [29] utilizes the magnitudes of the discrete values between training samples to divide the dataset and employs a sample selection threshold for class constraints, maintaining class balance in the selected clean set and making it robust to datasets with high noise levels. Additionally, some research has explored techniques such as estimating the noise transition matrix [23,[30][31][32][33][34], sample confidence estimation [35][36][37], and pseudo-labels [38] to handle noisy labels. These methods model, correct, or reassign weights to noisy labels to enhance the model's learning of true labels.…”

Class-Independent Regularization for Learning with Noisy Labels

“…The representative approaches of the former are small-loss criterion (Han et al 2018) (Yu et al 2019), which selects a human-defined proportion of small-loss samples as clean ones, and Gaussian Mixture Model (GMM) criterion (Li, Socher, and Hoi 2020), which fits GMM to the sample losses to model the distribution of clean and noisy samples. The representative approaches of the latter are prediction consistency, which partitions the training data into clean and noisy subsets based on the consistent predictions of two networks (Wei et al 2020) (Liang et al 2022) or two views (Yi and Huang 2021). However, the above methods rely heavily on the predictions from DNNs.…”

“…An active research direction is training DNNs with selected or reweighted training, where the challenge is to design a proper criterion for identifying clean samples. Existing criteria are mainly divided into two types: loss-based criterion (i.e., small-loss (Han et al 2018) (Yu et al 2019) and Gaussian Mixture Model (GMM) (Li, Socher, and Hoi 2020)) and consistency-based criterion (i.e., prediction consistency between two networks (Wei et al 2020) (Liang et al 2022) or two views (Yi and Huang 2021)). Although promising performance gains have been witnessed by employing these sample selection strategies, they heavily rely on the predictions from DNNs commonly trained with softmax cross-entropy loss.…”