Cost-Sensitive Learning of Deep Feature Representations From Imbalanced Data

Abstract

Abstract-Class imbalance is a common problem in the case of real-world object detection and classification tasks. Data of some classes is abundant making them an over-represented majority, and data of other classes is scarce, making them an underrepresented minority. This imbalance makes it challenging for a classifier to appropriately learn the discriminating boundaries of the majority and minority classes. In this work, we propose a cost-sensitive deep neural network which can automatically learn robust feature representations for both the majority and minority classes. During training, our learning procedure jointly optimizes the class-dependent costs and the neural network parameters. The proposed approach is applicable to both binary and multiclass problems without any modification. Moreover, as opposed to data level approaches, we do not alter the original data distribution which results in a lower computational cost during the training process. We report the results of our experiments on six major image classification datasets and show that the proposed approach significantly outperforms the baseline algorithms. Comparisons with popular data sampling techniques and cost-sensitive classifiers demonstrate the superior performance of our proposed method.

Tool wear classification using time series imaging and deep learning

“…If the training set is generated by randomly sampling from the complete dataset, it is likely that two-thirds of those samples are steady-state class. This class imbalance problem has been well documented in the literature [55][56][57]. Failure cases tend to be considerably less abundant than steady wear cases.…”

Proposal-Based Visual Tracking Using Spatial Cascaded Transformed Region Proposal Network

“…The loss function acts an important role in deep convolutional tracker by solving the data imbalance problem [ 26 ], though little attention had been paid to this kind of issue [ 27 ]. So far, the cost-sensitive loss [ 28 ] is proven to be an effective approach when suffering data imbalance. When pre-training the Siamese networks, Bertinetto et al [ 29 ] proposed to balance the loss of positives and negatives in order to improve the discriminative ability of the network.…”

Effective Melanoma Recognition Using Deep Convolutional Neural Network with Covariance Discriminant Loss

“…Nonetheless, over-sampling increases the probability of overfitting, and down-sampling risks from losing useful information. Algorithm-level methods focus on weighting the training samples, including loss-based methods and cost-sensitive methods [ 25 , 26 , 27 , 28 , 29 , 30 , 31 ]. For example, Lin et al [ 26 ] proposed focal loss to weight the hard samples.…”

“…In References [ 15 , 16 ], data imbalance is addressed with the cost-sensitive loss and Area Under the receiver operating characteristics Curve (AUC) statistics, which is directly derived from the statistic information of the input dataset. In References [ 20 , 21 , 22 , 23 , 24 , 25 , 26 , 27 , 28 , 29 , 30 , 31 , 32 , 33 , 34 , 35 , 36 ], data-level methods, algorithm-level methods, and hybrid methods were employed to address the data imbalance, boosting the performance to some extent. Hybrid methods inherit the advantages of data-level and algorithm-level methods [ 20 ].…”