BBW: a batch balance wrapper for training deep neural networks on extremely imbalanced datasets with few minority samples

Abstract

In recent years, Deep Neural Networks (DNNs) have achieved excellent performance on many tasks, but it is very difficult to train good models from imbalanced datasets. Creating balanced batches either by majority data down-sampling or by minority data up-sampling can solve the problem in certain cases. However, it may lead to learning process instability and overfitting. In this paper, we propose the Batch Balance Wrapper (BBW), a novel framework which can adapt a general DNN to be well trained from extremely imbalanced datasets with few minority samples. In BBW, two extra network layers are added to the start of a DNN. The layers prevent overfitting of minority samples and improve the expressiveness of the sample distribution of minority samples. Furthermore, Batch Balance (BB), a class-based sampling algorithm, is proposed to make sure the samples in each batch are always balanced during the learning process. We test BBW on three well-known extremely imbalanced datasets with few minority samples. The maximum imbalance ratio reaches 1167:1 with only 16 positive samples. Compared with existing approaches, BBW achieves better classification performance. In addition, BBW-wrapped DNNs are 16.39 times faster, relative to unwrapped DNNs. Moreover, BBW does not require data preprocessing or additional hyper-parameter tuning, operations that may require additional processing time. The experiments prove that BBW can be applied to common applications of extremely imbalanced data with few minority samples, such as the classification of EEG signals, medical images and so on.

Early prediction of distant metastasis in patients with uterine cervical cancer treated with definitive chemoradiotherapy by deep learning using pretreatment [18F]fluorodeoxyglucose positron emission tomography/computed tomography

“…According to the findings of our previous studies [8,13], the proportion of patients with distant metastases is relatively small. To ensure a balanced data set, we applied a batch balance wrapper during training [19]. Batch balance was also used to ensure that the samples in each batch were always balanced during the learning process (Appendix 1, Supplemental digital content 1, http://links.lww.com/NMC/A273).…”

Network Traffic Classification Based on SD Sampling and Hierarchical Ensemble Learning

“…Li et al [36] proposed a method called HM-loss cost, which pays more attention to the misclassifed samples in minority classes when calculating the loss. Hu et al [37] proposed a method of batch balancing datasets based on deep learning to ensure that the number of samples in each class is equal in each batch, so as to achieve the balance of datasets.…”