Cost Sensitive Learning in the Presence of Symmetric Label Noise

Abstract

In binary classification framework, we are interested in making cost sensitive label predictions in the presence of uniform/symmetric label noise. We first observe that 0-1 Bayes classifiers are not (uniform) noise robust in cost sensitive setting. To circumvent this impossibility result, we present two schemes; unlike the existing methods, our schemes do not require noise rate. The first one uses α-weighted γ-uneven margin squared loss function, lα,usq, which can handle cost sensitivity arising due to domain requirement (using user given α) or class imbalance (by tuning γ) or both. However, we observe that lα,usq Bayes classifiers are also not cost sensitive and noise robust. We show that regularized ERM of this loss function over the class of linear classifiers yields a cost sensitive uniform noise robust classifier as a solution of a system of linear equations. We also provide a performance bound for this classifier. The second scheme that we propose is a re-sampling based scheme that exploits the special structure of the uniform noise models and uses in-class probability η estimates. Our computational experiments on some UCI datasets with class imbalance show that classifiers of our two schemes are on par with the existing methods and in fact better in some cases w.r.t. Accuracy and Arithmetic Mean, without using/tuning noise rate. We also consider other cost sensitive performance measures viz., F measure and Weighted Cost for evaluation. If the noise is adversarial then, one can interpret not requiring the noise rates as a dominant strategy and cross-validation over noise rates as a search for a dominant strategy. Also, the required restriction of the hypothesis class to the linear class can be interpreted as an implicit form of regularization. As our re-sampling scheme requires estimates of η, we provide a detailed comparative study of various η estimation methods on synthetic datasets, w.r.t. half a dozen evaluation criterion. Also, we provide understanding on the interpretation of cost parameters α and γ using different synthetic data experiments.

GANs for learning from very high class conditional noisy labels

“…For imbalanced and symmetric label noise (ρ < 0.5) corrupted data, majority (minority) class continues to be in majority (minority) (Lemma 3 [16]).…”

“…In the last decade, label noise problem has gained a lot of attention from researchers due to its prevalence in various real life situations. [12,13,14,15,5,16] focus on effect of label noise problems on non-deep classification schemes and provide solutions which either consider a label noise robust loss function or modifies the loss function to make it robust. For deep learning schemes, [17] propose two algorithms called forward and backward loss correction to learn from label noise corrupted data, [18] identify label noise robust loss functions to be used by neural networks and [19] provide consistency results when the noise is instance dependent.…”

“…We would like to emphasize that our scheme is different from conventional WGAN in two ways: 1) data representation approach, i.e., the labels are appended in addition to the features and 2) the latent variable is not standard normal or uniform, instead it follows corrupted and hence unknown data distribution D. Also, unlike the existing work on learning with noisy labels, use of a generative model in label noise problem allows us to circumvent the problem of either estimating or tuning the noise rates. Even though [7,33,16] do not need the noise rates, the context is restricted either to symmetric label noise or non-high dimensional datasets. Next, we formally describe our scheme.…”

“…Also, radial basis oversampling proposed in [40] only demonstrates their results when the majority class doesn't flip to minority (or vice versa) and hence doesn't completely deal with the complexity of imbalanced label noise learning. [16] provide solution for the label noise (but only SLN) and imbalance problem without requiring the noise rate.…”

Making Deep Neural Networks Robust to Label Noise: Cross-Training With a Novel Loss Function

“…As for binary classification task, Masnadishirazi and Vasconcelos [26] proposed a savage loss based on boost algorithms and theoretically proved its robustness to outliers. Tripath and Hemachandra [27] have demonstrated that 0-1 loss is tolerant for uniform label noise. These loss functions, however, are neither appropriate for multi-classification scenarios nor training a very deep neural network.…”