Transfer and share: semi-supervised learning from long-tailed data

Wei, Tong; Liu, Qianyu; Shi, Jiang-Xin; Tu, Wei-Wei; Guo, Lan-Zhe

doi:10.1007/s10994-022-06247-z

Cited by 10 publications

(7 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Chang et al [25] proposed a joint resampling strategy, RIO. Wei [26] introduced open-sampling, a method that leverages out-of-distribution data to rebalance class priors and encourage separable representations. Yu et al [27] revived the use of balanced undersampling, achieving higher accuracy for worst-performing categories.…”

Section: Long-tailed Visual Recognitionmentioning

confidence: 99%

Long-Tailed Effect Study in Remote Sensing Semantic Segmentation Based on Graph Kernel Principles

Cui,

Feng,

Chen

et al. 2024

Remote Sensing

View full text Add to dashboard Cite

The performance of semantic segmentation in remote sensing, based on deep learning models, depends on the training data. A commonly encountered issue is the imbalanced long-tailed distribution of data, where the head classes contain the majority of samples while the tail classes have fewer samples. When training with long-tailed data, the head classes dominate the training process, resulting in poorer performance in the tail classes. To address this issue, various strategies have been proposed, such as resampling, reweighting, and transfer learning. However, common resampling methods suffer from overfitting to the tail classes while underfitting the head classes, and reweighting methods are limited in the extreme imbalanced case. Additionally, transfer learning tends to transfer patterns learned from the head classes to the tail classes without rigorously validating its generalizability. These methods often lack additional information to assist in the recognition of tail class objects, thus limiting performance improvements and constraining generalization ability. To tackle the abovementioned issues, a graph neural network based on the graph kernel principle is proposed for the first time. By leveraging the graph kernel, structural information for tail class objects is obtained, serving as additional contextual information beyond basic visual features. This method partially compensates for the imbalance between tail and head class object information without compromising the recognition accuracy of head classes objects. The experimental results demonstrate that this study effectively addresses the poor recognition performance of small and rare targets, partially alleviates the issue of spectral confusion, and enhances the model’s generalization ability.

show abstract

Section: Long-tailed Visual Recognitionmentioning

confidence: 99%

Long-Tailed Effect Study in Remote Sensing Semantic Segmentation Based on Graph Kernel Principles

Cui,

Feng,

Chen

et al. 2024

Remote Sensing

View full text Add to dashboard Cite

show abstract

“…Evaluation Measures Following (Yang et al 2021;Wang et al 2022), we use the below common metrics for OOD detection and ID classification: (1) FPR is the false posi- Implementation Details We compared our approach COCL with several existing OOD detection methods on long-tailed training sets, including classical methods MSP (Hendrycks and Gimpel 2016), OE (Hendrycks, Mazeika, and Dietterich 2018), EnergyOE (Liu et al 2020), and very recently published methods PASCL (Wang et al 2022), OS (Wei et al 2022a), Class Prior (Jiang et al 2023), andBERL (Choi, Jeong, andChoi 2023). The OCL method in our results is a baseline that is trained based on Eq.…”

Section: Experiments Experiments Settingsmentioning

confidence: 99%

“…Compared to OOD detection on balanced ID datasets, significantly less work has been done in the LTR scenarios. Recent studies (Wang et al 2022;Wei et al 2022a;Jiang et al 2023;Choi, Jeong, and Choi 2023) are among the seminal works exploring OOD detection in LTR. Current methods in this line focus on distinguishing OOD samples from ID samples using an approach called outlier exposure (OE) (Hendrycks, Mazeika, and Dietterich 2018) that fits auxiliary/pseudo OOD data to a prior distribution (e.g., uniform distribution) of ID data.…”

Section: Introductionmentioning

confidence: 99%

Out-of-Distribution Detection in Long-Tailed Recognition with Calibrated Outlier Class Learning

Miao,

Pang,

Bai

et al. 2024

AAAI

View full text Add to dashboard Cite

Existing out-of-distribution (OOD) methods have shown great success on balanced datasets but become ineffective in long-tailed recognition (LTR) scenarios where 1) OOD samples are often wrongly classified into head classes and/or 2) tail-class samples are treated as OOD samples. To address these issues, current studies fit a prior distribution of auxiliary/pseudo OOD data to the long-tailed in-distribution (ID) data. However, it is difficult to obtain such an accurate prior distribution given the unknowingness of real OOD samples and heavy class imbalance in LTR. A straightforward solution to avoid the requirement of this prior is to learn an outlier class to encapsulate the OOD samples. The main challenge is then to tackle the aforementioned confusion between OOD samples and head/tail-class samples when learning the outlier class. To this end, we introduce a novel calibrated outlier class learning (COCL) approach, in which 1) a debiased large margin learning method is introduced in the outlier class learning to distinguish OOD samples from both head and tail classes in the representation space and 2) an outlier-class-aware logit calibration method is defined to enhance the long-tailed classification confidence. Extensive empirical results on three popular benchmarks CIFAR10-LT, CIFAR100-LT, and ImageNet-LT demonstrate that COCL substantially outperforms existing state-of-the-art OOD detection methods in LTR while being able to improve the classification accuracy on ID data. Code is available at https://github.com/mala-lab/COCL.

show abstract

“…Mahalanobis distance-based score [120], Energy-based score [31,121,122], ReAct [123], GradNorm score [117], and non-parametric KNN-based score [124,125]. 2) Some works address the out-of-distribution detection problem by training-time regularization [3,31,[126][127][128][129][130][131][132][133][134][135]. For example, models are encouraged to give predictions with uniform distribution [3,126] or higher energies [31,[135][136][137][138] for outliers.…”

Section: Natural Robustness Of Machine Learningmentioning

confidence: 99%

“…2) Some works address the out-of-distribution detection problem by training-time regularization [3,31,[126][127][128][129][130][131][132][133][134][135]. For example, models are encouraged to give predictions with uniform distribution [3,126] or higher energies [31,[135][136][137][138] for outliers.…”

Section: Related Workmentioning

confidence: 99%

Natural robustness of machine learning in the open world

Wei¹

View full text Add to dashboard Cite

visor, Prof. Bo An. Without the great opportunity he provided, I could not have undertaken this Ph.D. journey, which has been a turning point in my life. During these years, he always maintains a gentle, patient, and kind style. He has not only provided a singularly suitable research environment with the greatest freedom, but also taught me the necessary knowledge in my research life, such as how to be professional and how to properly communicate with others. Words cannot express my gratitude to him. The experience in his group would definitely shape my future in many dimensions. I am so fortunate to become a Ph.D. student under his supervision and I wish to be such a nice professor like him in the future. I also had the pleasure of working with a group of talented people: Qingyu Guo,

show abstract

Transfer and share: semi-supervised learning from long-tailed data

Cited by 10 publications

References 18 publications

Long-Tailed Effect Study in Remote Sensing Semantic Segmentation Based on Graph Kernel Principles

Long-Tailed Effect Study in Remote Sensing Semantic Segmentation Based on Graph Kernel Principles

Out-of-Distribution Detection in Long-Tailed Recognition with Calibrated Outlier Class Learning

Natural robustness of machine learning in the open world

Contact Info

Product

Resources

About