2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW) 2019
DOI: 10.1109/iccvw.2019.00186
|View full text |Cite
|
Sign up to set email alerts
|

Video Representation Learning by Dense Predictive Coding

Abstract: Figure 1: Nearest Neighbour (NN) video clip retrieval on UCF101. Each row contains four video clips, a query clip and the top three retrievals using clip embeddings. To get the embedding, each video is passed to a 3D-ResNet18, average pooled to a single vector, and cosine similarity is used for retrieval. (a) Embeddings obtained by Dense Predictive Coding (DPC); (b) Embeddings obtained by using the inflated ImageNet pretrained weights. The DPC captures the semantics of the human action, rather than the scene a… Show more

Help me understand this report

Search citation statements

Order By: Relevance

Paper Sections

Select...
2
1
1
1

Citation Types

1
310
0

Year Published

2020
2020
2021
2021

Publication Types

Select...
5
2
1

Relationship

0
8

Authors

Journals

citations
Cited by 379 publications
(311 citation statements)
references
References 29 publications
1
310
0
Order By: Relevance
“…Our focus on supervised learning of static inputs was motivated by recent progress in this area. However, machine learning researchers have also been making rapid progress in unsupervised learning on temporal sequences in recent years [110,111]. We suspect that many of the same mechanisms we explored here, e.g.…”
Section: Discussionmentioning
confidence: 84%
“…Our focus on supervised learning of static inputs was motivated by recent progress in this area. However, machine learning researchers have also been making rapid progress in unsupervised learning on temporal sequences in recent years [110,111]. We suspect that many of the same mechanisms we explored here, e.g.…”
Section: Discussionmentioning
confidence: 84%
“…One promising set of candidates are predictive self-supervised loss functions [41, 21, 36]. Recent work has shown that self-supervised learning can produce similar results to supervised learning for the ventral pathway [57, 33, 25].…”
Section: Introductionmentioning
confidence: 99%
“…Here, we explore these issues using publicly available data from the Allen Brain Observatory [8], which provides recordings from a large number of areas in mouse visual cortex. We examine the ability of a self-supervised predictive loss function (contrastive predictive coding [41, 21]) to induce representations that match mouse visual cortex. When we train a network with a single pathway, we find that it possesses more ventral-like representations.…”
Section: Introductionmentioning
confidence: 99%
“…Self-supervised learning, a popular branch of unsupervised learning, has produced an impressive performance in many research fields such as computer vision, video processing and natural language processing [7,8,9,10,11,12,13], showing its great power of extracting high-level semantic features. However, only a few studies have discovered the power of SSL applying to biomedical signals.…”
Section: Related Workmentioning
confidence: 99%
“…To achieve this, SleepDPC learns representations by two principles. The first learning principle is predictive, which means to predict a slowly varying representation based on recent past observations [12,11]. Following the Slow Feature Analysis [16], we assume that an appropriate representation should slowly evolves over time (e.g.…”
Section: Learning Frameworkmentioning
confidence: 99%