Trustworthy Long-Tailed Classification

Abstract

Long-tailed classification poses a challenge due to its heavy imbalance in class probabilities and tailsensitivity risks with asymmetric misprediction costs. Recent attempts have used re-balancing loss and ensemble methods, but they are largely heuristic and depend heavily on empirical results, lacking theoretical explanation. Furthermore, existing methods overlook the decision loss, which characterizes different costs associated with tailed classes. This paper presents a general and principled framework from a Bayesian-decision-theory perspective, which unifies existing techniques including re-balancing and ensemble methods, and provides theoretical justifications for their effectiveness. From this perspective, we derive a novel objective based on the integrated risk and a Bayesian deep-ensemble approach to improve the accuracy of all classes, especially the "tail". Besides, our framework allows for task-adaptive decision loss which provides provably optimal decisions in varying task scenarios, along with the capability to quantify uncertainty. Finally, We conduct comprehensive experiments, including standard classification, tail-sensitive classification with a new False Head Rate metric, calibration, and ablation studies. Our framework significantly improves the current SOTA even on large-scale real-world datasets like ImageNet.

Revisiting Long-tailed Image Classification: Survey and Benchmarks with New Evaluation Metrics

“…Considering the real-world testing data may exhibit a distinct distribution compared to the training data, Zhang et al [41] learn multiple models under different distributions and combines them with weights generated via testing-time adaptation. For decreasing the computational cost, Li et al [13] propose to measure the uncertainty of each expert, and assign experts to each sample dynamically. Tang et al [25] utilize uniform intra-class data sampling and confidenceaware data sampling strategies to construct different training environments for learning features invariant to diversified attributes.…”

“…Existing methods concentrate on different procedures including data preparation, feature representation learning, objective function design, and class prediction to tackle this task. According to the focused procedures, they can be categorized into four types, i.e., data balancing [4,7], feature balancing [6], loss balancing [16,21,22], and prediction balancing [13,28,41]. Most of them train models with imbalanced subsets of existing datasets, such as CIFAR-10/100 [12], ImageNetLT [17], and iNaturalist [26].…”

Towards Long-Tailed Recognition for Graph Classification via Collaborative Experts

“…Long-tailed learning aims to alleviate the impact of class imbalance on model training, and there are currently three main strategies to address this practical problem: re-sampling [15], [16], [34]- [37], re-weighting [22], [38]- [41], and ensemble learning [18], [26], [42]- [44]. For the re-sampling group, SMOTE [15] aims to rebalance the data distribution by generating new samples and performing interpolation in the tail classes, which belongs to an over-sampling approach.…”

“…But plain re-weighting strategy could benefit classifier learning while hurting representation learning, because it may under-represent the head classes and cause unstable training [24], [25]. Different from the formers, ensemble learning [18], [26] combines multiple expert networks from a complementary perspective to obtain reliable and robust predictions, whose works have achieved satisfactory progress. Nevertheless, most current ensemble learning methods lack mutual supervision among different experts and knowledge transfer is also deficient.…”

Predicting Social Events with Multimodal Fusion of Spatial and Temporal Dynamic Graph Representations

“…By introducing dynamics, they can integrate reliable cues from multi-spectral data for around-the-clock applications (e.g., pedestrian detection in security surveillance and autonomous driving). Dynamic fusion has been used in diverse real-world multimodal applications, including multimodal classification (Han et al, 2021;Geng et al, 2021;Han et al, 2022b), regression (Ma et al, 2021), object detection (Li et al, 2022a;Zhang et al, 2019;Chen et al, 2022b) and semantic segmentation (Tian et al, 2020). While dynamic multimodal fusion shows excellent power in practice, theoretical understanding is notably lack in this field with the following fundamental open problem: Can we realize reliable multimodal fusion in practice with theoretical guarantee?…”