Complement Objective Training

Chen, Haoyun; Wang, Pei-Hsin; Liu, Chun-Hao; Chang, Shih-Chieh; Pan, Jia-Yu; Chen, Yuting; Wei, Wei; Juan, Da-Cheng

doi:10.48550/arxiv.1903.01182

Cited by 7 publications

(7 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Existing Methods. We compare the proposed method (CCE) with the following techniques: (i) empirical risk minimization (ERM): we train models with only standard softmax cross entropy; (ii) complement objective training (COT): there are two optimizers for training: one for softmax cross entropy and the other one for softmax complement entropy [8]; (iii) focal loss (FL): it uses sigmoid cross entropy with a modulating factor to concentrate on hard samples [24].…”

Section: Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Imbalanced Image Classification with Complement Cross Entropy

Kim¹,

Lee²,

Jeon³

2020

Preprint

View full text Add to dashboard Cite

Recently, deep learning models have achieved great success in computer vision applications, relying on large-scale class-balanced datasets. However, imbalanced class distributions still limit the wide applicability of these models due to degradation in performance. To solve this problem, we focus on the study of cross entropy: it mostly ignores output scores on wrong classes. In this work, we discover that neutralizing predicted probabilities on incorrect classes helps improve accuracy of prediction for imbalanced image classification. This paper proposes a simple but effective loss named complement cross entropy (CCE) based on this finding. Our loss makes the ground truth class overwhelm the other classes in terms of softmax probability, by neutralizing probabilities of incorrect classes, without additional training procedures. Along with it, this loss facilitates the models to learn key information especially from samples on minority classes. It ensures more accurate and robust classification results for imbalanced class distributions. Extensive experiments on imbalanced datasets demonstrate the effectiveness of our method compared to other state-of-the-art methods.

show abstract

Section: Methodsmentioning

confidence: 99%

“…It means that ̂ ( ) [ ≠ ] in the cross entropy is totally ignored, so inaccurately predicted probabilities may produce a cumulative error. To avoid such error, complement objective training (COT) was proposed by Chen et al, where the core idea is evenly suppressing softmax probabilities on incorrect classes during training [8].…”

Section: Introductionmentioning

confidence: 99%

Imbalanced Image Classification with Complement Cross Entropy

Kim¹,

Lee²,

Jeon³

2020

Preprint

View full text Add to dashboard Cite

show abstract

“…Differently, our PC loss with logit constraints is motivated by the predictive behavior of CNN on adversarial samples from the probability perspective, which avoids this issue by learning probabilistically compact features without geometric assumptions. The work that is closest to ours is (Chen et al 2019b) that encourages the predicted probabilities of false classes to be equally distributed, whereas our PC loss directly enlarges the gap of probabilities between true class and the first several most probable false classes.…”

Section: Related Workmentioning

confidence: 96%

Improving Adversarial Robustness via Probabilistically Compact Loss with Logit Constraints

Deng

et al. 2021

AAAI

View full text Add to dashboard Cite

Convolutional neural networks (CNNs) have achieved state-of-the-art performance on various tasks in computer vision. However, recent studies demonstrate that these models are vulnerable to carefully crafted adversarial samples and suffer from a significant performance drop when predicting them. Many methods have been proposed to improve adversarial robustness (e.g., adversarial training and new loss functions to learn adversarially robust feature representations). Here we offer a unique insight into the predictive behavior of CNNs that they tend to misclassify adversarial samples into the most probable false classes. This inspires us to propose a new Probabilistically Compact (PC) loss with logit constraints which can be used as a drop-in replacement for cross-entropy (CE) loss to improve CNN's adversarial robustness. Specifically, PC loss enlarges the probability gaps between true class and false classes meanwhile the logit constraints prevent the gaps from being melted by a small perturbation. We extensively compare our method with the state-of-the-art using large scale datasets under both white-box and black-box attacks to demonstrate its effectiveness. The source codes are available at https://github.com/xinli0928/PC-LC.

show abstract

“…In addition, Chen proposed complement entropy loss [32], which suppresses misclassification and achieves error neutralization by exploiting the complement information of samples in this class, thus improving the confidence level. Because the minority class is richer in error information, Kim et al [33] used complement entropy to improve the class imbalance problem.…”

Section: Manuscript Clear Copymentioning

confidence: 99%

Adaptive Margin Aware Complement-Cross Entropy Loss for Improving Class Imbalance in Multi-View Sleep Staging Based on EEG Signals

Miao

Yao

Zhao

2022

IEEE Trans. Neural Syst. Rehabil. Eng.

View full text Add to dashboard Cite

Sleep data are typically characterized by class imbalance, which can cause the model to be overly biased toward frequent classes, resulting in low accuracy of minority class classification. However, the minority class of sleep staging has important value in diagnosing certain disorders, such as an N1 Stage that is too short indicating possible hypersomnia or narcolepsy. To address this problem, we propose a multi-view CNN model based on adaptive margin-aware loss. A novel margin-aware factor that considers the relative sample sizes of both frequent and minority classes can improve the overfitting of minority classes by increasing the regularization strength of minority classes without changing the sample size to maximize the decision margins of minority classes. On this basis, we propose margin-aware cross-entropy and margin-aware complement entropy loss, respectively. Margin-aware complement entropy can be achieved to increase the regularization for minority classes while neutralizing errors for minority classes, thus improving the classification accuracy for minority classes. Finally, the synergy of margin-aware complement entropy and cross-entropy is realized in an adaptive way to improve the sleep staging classification accuracy. We tested on three sleep datasets and compared them with the state-of-the-art, and the results demonstrate that our proposed algorithm not only improves the accuracy of sleep staging in general, but also improves the minority classes to a greater extent.

show abstract

Complement Objective Training

Cited by 7 publications

References 14 publications

Imbalanced Image Classification with Complement Cross Entropy

Imbalanced Image Classification with Complement Cross Entropy

Improving Adversarial Robustness via Probabilistically Compact Loss with Logit Constraints

Adaptive Margin Aware Complement-Cross Entropy Loss for Improving Class Imbalance in Multi-View Sleep Staging Based on EEG Signals

Contact Info

Product

Resources

About