Asymmetric Loss For Multi-Label Classification

Ridnik, Tal; Ben-Baruch, Emanuel; Zamir, Nadav; Noy, Asaf; Friedman, Itamar; Protter, Matan; Zelnik-Manor, Lihi

doi:10.1109/iccv48922.2021.00015

Cited by 319 publications

(171 citation statements)

References 20 publications

Supporting

Mentioning

171

Contrasting

Order By: Relevance

“…We now evaluate and compare such representations against state-of-the-art DML models to understand if generic representations that are readily available nowadays actually pose an alternative to explicit application of DML. We select state-of-the-art self-supervision models SwAV [3] (ResNet50 backbone), CLIP [51] trained via natural language supervision on a large dataset of 400 million image and sentence pairs (VisionTransformer [10] backbone), BiT(-M) [30], which trains a ResNet50-V2 [30] on both the standard ImageNet [7] (1 million training samples) and the ImageNet-21k dataset [7,54] with 14 million training samples and over 21 thousand classes, an EfficientNet-B0 [66] trained on ImageNet, and a standard baseline ResNet50 network trained on ImageNet. We note that none of these representations have been additionally adapted to the benchmark sets and only the pretrained representations are being evaluated, in contrast to the DML approaches which have been trained on the respective train splits.…”

Section: Generic Representations Versus Deep Metric Learningmentioning

confidence: 99%

Characterizing Generalization under Out-Of-Distribution Shifts in Deep Metric Learning

Milbich¹,

Roth²,

Sinha³

et al. 2021

Preprint

View full text Add to dashboard Cite

Deep Metric Learning (DML) aims to find representations suitable for zero-shot transfer to a priori unknown test distributions. However, common evaluation protocols only test a single, fixed data split in which train and test classes are assigned randomly. More realistic evaluations should consider a broad spectrum of distribution shifts with potentially varying degree and difficulty. In this work, we systematically construct train-test splits of increasing difficulty and present the ooDML benchmark to characterize generalization under out-of-distribution shifts in DML. ooDML is designed to probe the generalization performance on much more challenging, diverse train-to-test distribution shifts. Based on our new benchmark, we conduct a thorough empirical analysis of state-of-the-art DML methods. We find that while generalization tends to consistently degrade with difficulty, some methods are better at retaining performance as the distribution shift increases. Finally, we propose few-shot DML as an efficient way to consistently improve generalization in response to unknown test shifts presented in ooDML. Code available here: https://github.com/Confusezius/Characterizing_Generalization_in_ DeepMetricLearning.x Work done while at IWR, Heidelberg University. * Equal contribution, alphabetical order. † Equal supervision. 1 E.g. first and second half of alphabetically sorted classnames in CUB200-2011[71] and CARS196[31].Preprint. Under review.

show abstract

Section: Generic Representations Versus Deep Metric Learningmentioning

confidence: 99%

Characterizing Generalization under Out-Of-Distribution Shifts in Deep Metric Learning

Milbich¹,

Roth²,

Sinha³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Its ability to quickly fine-tune to a smaller, downstream dataset, generalizing even in a few-shot regime, makes it an attractive backbone for tasks such as out-of-distribution (OOD) detection. In this paper, we use ViT pre-trained on ImageNet-21k [Ridnik et al, 2021].…”

Section: Pre-training Neural Networkmentioning

confidence: 99%

Exploring the Limits of Out-of-Distribution Detection

Fort,

Ren,

Lakshminarayanan

2021

Preprint

View full text Add to dashboard Cite

Near out-of-distribution detection (OOD) is a major challenge for deep neural networks. We demonstrate that large-scale pre-trained transformers can significantly improve the state-of-the-art (SOTA) on a range of near OOD tasks across different data modalities. For instance, on CIFAR-100 vs CIFAR-10 OOD detection, we improve the AUROC from 85% (current SOTA) to more than 96% using Vision Transformers pre-trained on ImageNet-21k. On a challenging genomics OOD detection benchmark, we improve the AUROC from 66% to 77% using transformers and unsupervised pre-training. To further improve performance, we explore the few-shot outlier exposure setting where a few examples from outlier classes may be available; we show that pre-trained transformers are particularly well-suited for outlier exposure, and that the AUROC of OOD detection on CIFAR-100 vs CIFAR-10 can be improved to 98.7% with just 1 image per OOD class, and 99.46% with 10 images per OOD class. For multi-modal image-text pre-trained transformers such as CLIP, we explore a new way of using just the names of outlier classes as a sole source of information without any accompanying images, and show that this outperforms previous SOTA on standard vision OOD benchmark tasks. * Equal contribution.Preprint. Under review.

show abstract

“…To demonstrate the general applicability of our TD module across different tasks and assess its capability as a robust feature extractor, we evaluate its performance as a backbone for existing state-of-the-art methods for fine-grained classification and multi-label object classification. We use the "Weakly Supervised Data Augmentation" method [18] for fine-grained classification and "Asymmetric loss" method [29] for multi-label classification, and simply replace the model backbone in both methods with our pretrained ImageNet-1k models. For fine-grained classification, we consider the Caltech-birds (CUB) and Stanford Dogs (Dogs) datasets and assess models on top-1 validation accuracy.…”

Section: Fine-grained and Multi-label Classificationmentioning

confidence: 99%

TDAM: Top-Down Attention Module for Contextually Guided Feature Selection in CNNs

Jaiswal¹,

Fernando²,

Tan³

2021

Preprint

View full text Add to dashboard Cite

Attention modules for Convolutional Neural Networks (CNNs) are an effective method to enhance performance of networks on multiple computer-vision tasks. While many works focus on building more effective modules through appropriate modelling of channel-, spatial-and self-attention, they primarily operate in a feedfoward manner. Consequently, the attention mechanism strongly depends on the representational capacity of a single input feature activation, and can benefit from incorporation of semanticallyricher higher-level activations that can specify "what and where to look" through top-down information flow. Such feedback connections are also prevalent in the primate visual cortex and recognized by neuroscientists as a key component in primate visual attention.Accordingly, in this work, we propose a lightweight topdown (TD) attention module that iteratively generates a "visual searchlight" to perform top-down channel and spatial modulation of its inputs and consequently outputs more selective feature activations at each computation step. Our experiments indicate that integrating TD in CNNs enhances their performance on ImageNet-1k classification and outperforms prominent attention modules while being more parameter and memory efficient. Further, our models are more robust to changes in input resolution during inference and learn to "shift attention" by localizing individual objects or features at each computation step without any explicit supervision. This capability results in 5% improvement for ResNet50 on weakly-supervised object localization besides improvements in fine-grained and multi-label classification.

show abstract

Asymmetric Loss For Multi-Label Classification

Cited by 319 publications

References 20 publications

Characterizing Generalization under Out-Of-Distribution Shifts in Deep Metric Learning

Characterizing Generalization under Out-Of-Distribution Shifts in Deep Metric Learning

Exploring the Limits of Out-of-Distribution Detection

TDAM: Top-Down Attention Module for Contextually Guided Feature Selection in CNNs

Contact Info

Product

Resources

About