Parametric Contrastive Learning

Cui, Jiequan; Zhong, Zhisheng; Liu, Shu; Yu, Bei; Jia, Jiaya

doi:10.1109/iccv48922.2021.00075

Cited by 182 publications

(121 citation statements)

References 25 publications

Supporting

Mentioning

121

Contrasting

Order By: Relevance

“…They show that transforming the classification head, as opposed to re-training it, performs better. PaCo [7], the current state-of-the-art for longtail classification, combines learnable logit adjustment with contrastive learning [40]. Despite their simplicity, adjusted logit methods (LACE, LDAM, LADE) remain strong solutions to the long tail problem, typically achieving within 1-2% top-1 accuracy of state-of-the-art ensemble approaches (see Table 1, Table 2).…”

Section: Related Workmentioning

confidence: 99%

“…Base approaches are largely variants of the same core idea-that of "adjustment", where the learner is encouraged to focus on the tail of the distribution. This can be achieved implicitly, via over/under-weighting samples during training [3,13,20,23] or cluster-based sampling [7], or explicitly via logit [11,38,58] or loss [22,38] modification. Such approaches largely focus on consistency, ensuring minimizing the training loss corresponds to a minimal error on the known, balanced, test distribution.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Retrieval Augmented Classification for Long-Tail Visual Recognition

Long¹,

Yin²,

Ajanthan³

et al. 2022

Preprint

View full text Add to dashboard Cite

We introduce Retrieval Augmented Classification (RAC), a generic approach to augmenting standard image classification pipelines with an explicit retrieval module. RAC consists of a standard base image encoder fused with a parallel retrieval branch that queries a non-parametric external memory of pre-encoded images and associated text snippets. We apply RAC to the problem of long-tail classification and demonstrate a significant improvement over previous state-of-the-art on Places365-LT and iNaturalist-2018 (14.5% and 6.7% respectively), despite using only the training datasets themselves as the external information source. We demonstrate that RAC's retrieval module, without prompting, learns a high level of accuracy on tail classes. This, in turn, frees the base encoder to focus on common classes, and improve its performance thereon. RAC represents an alternative approach to utilizing large, pretrained models without requiring fine-tuning, as well as a first step towards more effectively making use of external memory within common computer vision architectures.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Retrieval Augmented Classification for Long-Tail Visual Recognition

Long¹,

Yin²,

Ajanthan³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Zhang et al [33] and Hong et al [14] propose recalibration and postprocessing strategies to address the label distribution shift between training and test datasets. Zhong et al [35] rely on label smoothing and mixup strategies, while Cui et al [7] combine cross-entropy with supervised contrastive learning, achieving stronger representations at the cost of more expensive training. Finally, closest to our work, Samuel et al [26] propose to rely on a prototype-based auxiliary loss, where class prototypes are computed at the beginning of every epoch during training.…”

Section: Related Workmentioning

confidence: 99%

“…Our work focuses on this issue, and is not tied to a rebalancing strategy. Therefore, it provides complementary benefits to approaches that focus on backbone training improvements [7,24,26], as discussed above.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Long-tail Recognition via Compositional Knowledge Transfer

Parisot¹,

Esperança²,

McDonagh³

et al. 2021

Preprint

View full text Add to dashboard Cite

In this work, we introduce a novel strategy for long-tail recognition that addresses the tail classes' few-shot problem via training-free knowledge transfer. Our objective is to transfer knowledge acquired from information-rich common classes to semantically similar, and yet data-hungry, rare classes in order to obtain stronger tail class representations. We leverage the fact that class prototypes and learned cosine classifiers provide two different, complementary representations of class cluster centres in feature space, and use an attention mechanism to select and recompose learned classifier features from common classes to obtain higher quality rare class representations. Our knowledge transfer process is training free, reducing overfitting risks, and can afford continual extension of classifiers to new classes. Experiments show that our approach can achieve significant performance boosts on rare classes while maintaining robust common class performance, outperforming directly comparable state-of-the-art models.

show abstract

MA‐CAT: Misclassification‐Aware Contrastive Adversarial Training

Zhi,

Yu,

et al. 2024

Advanced Intelligent Systems

View full text Add to dashboard Cite

Vulnerability to adversarial examples poses a significant challenge to the secure application of deep neural networks. Adversarial training and its variants have shown great potential in addressing this problem. However, such approaches, which directly optimize the decision boundary, often result in overly complex adversarial decision boundaries that are detrimental to generalization. To deal with this issue, a novel plug‐and‐play method known as Misclassification‐Aware Contrastive Adversarial Training (MA‐CAT) from the perspective of data distribution optimization is proposed. MA‐CAT leverages supervised decoupled contrastive learning to cluster nature examples within the same class in the logit space, indirectly increasing the margins of examples. Moreover, by taking into account the varying difficulty levels of adversarial training for different examples, MA‐CAT adaptively customizes the strength of adversarial training for each example using an instance‐wise misclassification‐aware adaptive temperature coefficient. Extensive experiments on the CIFAR‐10, CIFAR‐100, and SVHN datasets demonstrate that MA‐CAT can be easily integrated into existing models and significantly improves robustness with minimal computational cost.

show abstract

Parametric Contrastive Learning

Cited by 182 publications

References 25 publications

Retrieval Augmented Classification for Long-Tail Visual Recognition

Retrieval Augmented Classification for Long-Tail Visual Recognition

Long-tail Recognition via Compositional Knowledge Transfer

MA‐CAT: Misclassification‐Aware Contrastive Adversarial Training

Contact Info

Product

Resources

About