When Does Contrastive Visual Representation Learning Work?

Cole, Elijah; Yang, Xuan; Wilber, Kimberly; Aodha, Oisin Mac; Belongie, Serge

doi:10.1109/cvpr52688.2022.01434

Cited by 66 publications

(22 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Nevertheless, a recent SSL study also supports our claim and sheds some light on this issue as follows: [41] claims that current self-supervised methods learn representations that can easily disambiguate coarse-grained visual concepts like those in ImageNet. However, as the granularity of the concepts becomes finer, self-supervised performance lags further behind supervised baselines.…”

Section: Discussionsupporting

confidence: 76%

Saliency Can Be All You Need In Contrastive Self-Supervised Learning

Kocaman¹,

Shir²,

Bäck³

et al. 2022

Preprint

View full text Add to dashboard Cite

We propose an augmentation policy for Contrastive Self-Supervised Learning (SSL) in the form of an already established Salient Image Segmentation technique entitled Global Contrast based Salient Region Detection. This detection technique, which had been devised for unrelated Computer Vision tasks, was empirically observed to play the role of an augmentation facilitator within the SSL protocol. This observation is rooted in our practical attempts to learn, by SSL-fashion, aerial imagery of solar panels, which exhibit challenging boundary patterns. Upon the successful integration of this technique on our problem domain, we formulated a generalized procedure and conducted a comprehensive, systematic performance assessment with various Contrastive SSL algorithms subject to standard augmentation techniques. This evaluation, which was conducted across multiple datasets, indicated that the proposed technique indeed contributes to SSL. We hypothesize whether salient image segmentation may suffice as the only augmentation policy in Contrastive SSL when treating downstream segmentation tasks.

show abstract

Section: Discussionsupporting

confidence: 76%

Saliency Can Be All You Need In Contrastive Self-Supervised Learning

Kocaman¹,

Shir²,

Bäck³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…We attribute this to the smaller amount of domain‐specific unlabeled image data used for SSL pretraining. Evidence suggests that performance of SSL models increases with the availability of larger unlabeled datasets (Cole et al., 2021).…”

Section: Resultsmentioning

confidence: 99%

Plant phenotyping with limited annotation: Doing more with less

Nagasubramanian

Singh

Sarkar

et al. 2022

The Plant Phenome Journal

View full text Add to dashboard Cite

Deep learning (DL) methods have transformed the way we extract plant traits—both under laboratory as well as field conditions. Evidence suggests that “well‐trained” DL models can significantly simplify and accelerate trait extraction as well as expand the suite of extractable traits. Training a DL model typically requires the availability of copious amounts of annotated data; however, creating large‐scale annotated dataset requires nontrivial efforts, time, and resources. This limitation has become a major bottleneck in deploying DL tools in practice. Self‐supervised learning (SSL) methods give exciting solution to this problem, as these methods use unlabeled data to produce pretrained models for subsequent fine‐tuning on labeled data and have demonstrated superior transfer learning performance on down‐stream classification tasks. We investigated the application of SSL methods for plant stress classification using few labels. We select a plant stress classification problem to test the effectiveness of SSL, as it is a fundamentally challenging problem due to (a) disease classification which depends on the abnormalities in a small number of pixels, (b) high data imbalance across different classes, and (c) fewer annotated and available plant stress images than in other domains. We compared seven SSL approaches spanning four broad classes of SSL methods on soybean [Glycine max L. (Merr.)] plant stress dataset and report that pretraining on unlabeled plant stress images significantly outperforms transfer learning methods using random initialization for plant stress classification. In summary, SSL‐based model initialization and data curation improves annotation efficiency for plant stress classification tasks and will circumvent data annotation challenges associated with DL methods.

show abstract

“…For example, D w was designed to be large enough for self-supervised learning, which has received surprisingly little attention in the WSOL community [9]. We are also interested in using iNatLoc500 to study whether self-supervised learning methods can be improved by using WSOL methods to select crops [40], especially in the context of fine-grained data [15]. For the object detection community, the clean boxes in iNatLoc500 can (i) serve as a test set for object detectors trained on the noisy iNat17 boxes, (ii) be used to study the problem of learning multi-instance detectors from one box per image, and (iii) be used to analyze the role of label granularity in object detection.…”

Section: Discussionmentioning

confidence: 99%

On Label Granularity and Object Localization

Cole

Wilber

Horn

et al. 2022

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

Weakly supervised object localization (WSOL) aims to learn representations that encode object location using only image-level category labels. However, many objects can be labeled at different levels of granularity. Is it an animal, a bird, or a great horned owl? Which image-level labels should we use? In this paper we study the role of label granularity in WSOL. To facilitate this investigation we introduce iNat-Loc500, a new large-scale fine-grained benchmark dataset for WSOL. Surprisingly, we find that choosing the right training label granularity provides a much larger performance boost than choosing the best WSOL algorithm. We also show that changing the label granularity can significantly improve data efficiency.

show abstract

When Does Contrastive Visual Representation Learning Work?

Cited by 66 publications

References 38 publications

Saliency Can Be All You Need In Contrastive Self-Supervised Learning

Saliency Can Be All You Need In Contrastive Self-Supervised Learning

Plant phenotyping with limited annotation: Doing more with less

On Label Granularity and Object Localization

Contact Info

Product

Resources

About