2022
DOI: 10.48550/arxiv.2205.06926
|View full text |Cite
Preprint
|
Sign up to set email alerts
|

Toward a Geometrical Understanding of Self-supervised Contrastive Learning

Help me understand this report

Search citation statements

Order By: Relevance

Paper Sections

Select...
1
1

Citation Types

0
2
0

Year Published

2022
2022
2023
2023

Publication Types

Select...
1
1

Relationship

1
1

Authors

Journals

citations
Cited by 2 publications
(2 citation statements)
references
References 0 publications
0
2
0
Order By: Relevance
“…Specifically, while all the models have a low-affinity value between these two directions (orthogonality), both simCLR versions have a high-affinity score (≈ .8) meaning that the subspaces spanning the semantic direction and the augmented direction are more aligned. This shows that the dimensional collapse effect observed and analyzed in SimCLR's projection head [19][20][21] appears to impact the backbone encoder representation as well. Specifically, simCLR-v1 and simCLR-v2 are the only models projecting the augmentation manifold onto the data manifold such that they are hardly distinguishable.…”
Section: Geometry Of Ssl Modelsmentioning
confidence: 81%
See 1 more Smart Citation
“…Specifically, while all the models have a low-affinity value between these two directions (orthogonality), both simCLR versions have a high-affinity score (≈ .8) meaning that the subspaces spanning the semantic direction and the augmented direction are more aligned. This shows that the dimensional collapse effect observed and analyzed in SimCLR's projection head [19][20][21] appears to impact the backbone encoder representation as well. Specifically, simCLR-v1 and simCLR-v2 are the only models projecting the augmentation manifold onto the data manifold such that they are hardly distinguishable.…”
Section: Geometry Of Ssl Modelsmentioning
confidence: 81%
“…For instance, [19] analyzes contrastive loss functions and the dimensional collapse problem. [20] also analyzes contrastive losses and describes the effect of augmentation strength as well as the importance of non-linear projection head. [21] quantifies the importance of data augmentations in a contrastive SSL model via a distance-based approach.…”
Section: Introductionmentioning
confidence: 99%