Learning Imbalanced Datasets With Maximum Margin Loss

Abstract

A learning algorithm referred to as Maximum Margin (MM) is proposed for considering the class-imbalance data learning issue: the deep model tends to predict the majority classes rather than the minority ones. For better generalization on the minority classes, the proposed Maximum Margin (MM) loss function is newly designed by minimizing a margin-based generalization bound through the shifting decision bound. As a prior study, the theoretically principled label-distributionaware margin (LDAM) loss had been successfully applied with classical strategies such as re-weighting or re-sampling. However, the maximum margin loss function has not been investigated so far. In this study, we evaluate the two types of hard maximum margin-based decision boundary shift with training schedule on artificially imbalanced CIFAR-10/100 and show the effectiveness.

Long-tailed Recognition by Routing Diverse Distribution-Aware Experts

“…Knowledge transfer. Another major direction to learning balance is transferring knowledge from head to tail classes (Liu et al, 2019;Zhu & Yang, 2020;Kang et al, 2020;Jamal et al, 2020;Wang et al, 2017). For example, the representative approaches in this direction, OLTR (Liu et al, 2019) and inflated memory (Zhu & Yang, 2020), harness memory banks to store and transfer mid-and high-level features from head classes to enhance tail classes' feature generalization.…”

“…Long-tailed recognition is usually handled either by class re-balancing/re-weighting strategies which give more importance to tail instances (Cao et al, 2019;Kang et al, 2020;Liu et al, 2019) or by grouping methods, where long-tailed data are separated into groups by their class frequencies and models focusing on each individual group are combined to form a multi-expert framework (Zhou et al, 2020;Xiang & Ding, 2020). However, all these methods generally gain on tail classes at the cost of performance loss on head classes.…”

“…Fig. 1a uses the standard cross-entropy (CE) classifier as a performance reference and compares three well-known long-tail methods, cRT and τ -norm (Kang et al, 2020) and LDAM (Cao et al, 2019), with our proposed method. The former adopts a two-stage optimization, first representation…”

“…The state-of-the-art (SOTA) methods on iNaturalist (Van Horn et al, 2018) are cRT and ⌧ -norm (Kang et al, 2020) and BBN (Zhou et al, 2020). The former belongs to the re-balancing type with a two-stage optimization for learning a good representation and classifier, whereas the latter belongs to the multi-expert type with two experts focusing on head and tail classes.…”

“…RIDE delivers 5%∼7% higher accuracies than the current SOTA methods on CIFAR100-LT, ImageNet-LT (Liu et al, 2019) and iNaturalist (Van Horn et al, 2018). RIDE is also a universal framework that can be applied to different backbone networks for improving existing long-tail algorithms such as focal loss (Lin et al, 2017), LDAM (Cao et al, 2019), τ -norm (Kang et al, 2020).…”