Temporal Alignment for History Representation in Reinforcement Learning

Ermolov, Aleksandr; Sangineto, Enver; Sebe, Nicu

doi:10.1109/icpr56361.2022.9956553

Cited by 20 publications

(25 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We use by default an ImageNet (Deng et al, 2009) pretrained Vision Transformer (ViT-B) (Dosovitskiy et al, 2021) as the image view teacher, and we use the text encoder from CLIP (Radford et al, 2021) as the text view teacher. The image and text teacher encoders are frozen during pretraining, and Smooth 1 -based positive-only distillation Ermolov et al, 2021) is used. Fig.…”

Section: Methodsmentioning

confidence: 99%

“…is the distance function defined in some metric space C. To avoid representation collapsing (He et al, 2020), InfoMax-Principle (Hjelm et al, 2019;Bachman et al, 2019) based metrics like MINE (Belghazi et al, 2018), InfoNCE (van den Oord et al, 2018 are often used (He et al, 2020;Tian et al, 2020c;Zhang et al, 2022a). For methods using positive-only transformations, the metric function can be feature correlation measurement (Zbontar et al, 2021), 2 distance (Ermolov et al, 2021), or cosine similarity (Grill et al, 2020;.…”

Section: Knowledge Distillation: a Unified View Of Generative And Con...mentioning

confidence: 99%

See 1 more Smart Citation

Contrast with Reconstruct: Contrastive 3D Representation Learning Guided by Generative Pretraining

Qi¹,

Dong²,

Fan³

et al. 2023

Preprint

View full text Add to dashboard Cite

Mainstream 3D representation learning approaches are built upon contrastive or generative modeling pretext tasks, where great improvements in performance on various downstream tasks have been achieved. However, by investigating the methods of these two paradigms, we find that (i) contrastive models are data-hungry that suffer from a representation over-fitting issue; (ii) generative models have a data filling issue that shows inferior data scaling capacity compared to contrastive models. This motivates us to learn 3D representations by sharing the merits of both paradigms, which is non-trivial due to the pattern difference between the two paradigms. In this paper, we propose contrast with reconstruct (RECON) that unifies these two paradigms. RECON is trained to learn from both generative modeling teachers and cross-modal contrastive teachers through ensemble distillation, where the generative student guides the contrastive student. An encoder-decoder style RECON-block is proposed that transfers knowledge through cross attention with stop-gradient, which avoids pretraining over-fitting and pattern difference issues. RECON achieves a new state-of-the-art in 3D representation learning, e.g., 91.26% accuracy on ScanObjectNN. Codes will be released at https://github.com/qizekun/ReCon.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Knowledge Distillation: a Unified View Of Generative And Con...mentioning

confidence: 99%

Contrast with Reconstruct: Contrastive 3D Representation Learning Guided by Generative Pretraining

Qi¹,

Dong²,

Fan³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Before delving into the details of the constraints, we start with the definition of autocovariance matrix: Whitening Normalization. Whitening normalization used in self-supervised computer vision tasks can scatter the batch samples to avoid degenerating embedding solutions collapsed onto a few dimensions or into a single point [47], with more feature or embedding decorrelation effect than Batch Normalization (BN) [48], even though BN also boosts the whitening effectiveness additionally [49]. In this work, we design a whitening normalization by employing Zerophase Component Analysis (ZCA) sphering [49], [50] to decorrelate any two graph signals s i and s j in the node embeddings H (l) .…”

Section: Three Constraints Derived From Graph Signal Decorrelationmentioning

confidence: 99%

Self-Supervised Node Representation Learning via Node-to-Neighbourhood Alignment

Wang¹,

Yan²,

Wang³

2023

Preprint

View full text Add to dashboard Cite

Self-supervised node representation learning aims to learn node representations from unlabelled graphs that rival the supervised counterparts. The key towards learning informative node representations lies in how to effectively gain contextual information from the graph structure. In this work, we present simple-yet-effective self-supervised node representation learning via aligning the hidden representations of nodes and their neighbourhood. Our first idea achieves such node-to-neighbourhood alignment by directly maximizing the mutual information between their representations, which, we prove theoretically, plays the role of graph smoothing. Our framework is optimized via a surrogate contrastive loss and a Topology-Aware Positive Sampling (TAPS) strategy is proposed to sample positives by considering the structural dependencies between nodes, which enables offline positive selection. Considering the excessive memory overheads of contrastive learning, we further propose a negative-free solution, where the main contribution is a Graph Signal Decorrelation (GSD) constraint to avoid representation collapse and over-smoothing. The GSD constraint unifies some of the existing constraints and can be used to derive new implementations to combat representation collapse. By applying our methods on top of simple MLP-based node representation encoders, we learn node representations that achieve promising node classification performance on a set of graph-structured datasets from small-to large-scale.

show abstract

“…BYOL [5] only uses positive examples' representations to compute mean squared error and gets comparable performance. W-MSE [21] uses a whitening transform to avoid degenerate solutions. Some contrastive models try to construct loss function from the perspective of the feature.…”

Section: Contrastive Learningmentioning

confidence: 99%

DPCL: Constrative Representation Learning with Differential Privacy

Yan

et al. 2022

Preprint

View full text Add to dashboard Cite

With the proliferation of unlabeled data, increasing efforts have been devoted to unsupervised learning. As one of the most representative branches of unsupervised learning, contrastive learning has made great progress with its high efficiency. Unfortunately, privacy threats on con-trastive learning have become sophisticated, making it imperative to develop effective technologies that are able to deal with such threats. To alleviate the privacy issue in contrastive learning, we propose some novel techniques based on differential privacy, which aim at reducing the high sensitivity of gradient in the private training caused by interactive contrastive learning. Specifically, we add differentially private protection to the connection point related to different per-example gradients, which decreases the sensitivity of the gradients significantly. Our experiments on SimCLR and Barlow Twins demonstrate the superiority of our approach with higher accuracy under the same privacy protection.

show abstract

Temporal Alignment for History Representation in Reinforcement Learning

Cited by 20 publications

References 14 publications

Contrast with Reconstruct: Contrastive 3D Representation Learning Guided by Generative Pretraining

Contrast with Reconstruct: Contrastive 3D Representation Learning Guided by Generative Pretraining

Self-Supervised Node Representation Learning via Node-to-Neighbourhood Alignment

DPCL: Constrative Representation Learning with Differential Privacy

Contact Info

Product

Resources

About