Incorporating Scalability in Unsupervised Spatio- Temporal Feature Learning

Abstract

Deep neural networks are efficient learning machines which leverage upon a large amount of manually labeled data for learning discriminative features. However, acquiring substantial amount of supervised data, especially for videos can be a tedious job across various computer vision tasks. This necessitates learning of visual features from videos in an unsupervised setting. In this paper, we propose a computationally simple, yet effective, framework to learn spatio-temporal feature embedding from unlabeled videos. We train a Convolutional 3D Siamese network using positive and negative pairs mined from videos under certain probabilistic assumptions. Experimental results on three datasets demonstrate that our proposed framework is able to learn weights which can be used for same as well as cross dataset and tasks.

Exploiting Temporal Coherence for Self-Supervised One-shot Video Re-identification

“…We investigate the contributions of the temporal losses to the re-identification TCPL (L) EUG [30] One-Shot Prog. [29] (b) Fig. 4.…”

“…(1) using the unlabeled data efficiently is important, ( TCPL EUG [30] One-Shot Progressive [29] (b) performance. In order to do that, we performed experiments with different values of λ (higher value indicates larger weight on the temporal losses) and present the results on the DukeMTMC-VideoReID dataset in Fig.…”

“…Intra-sequence temporal consistency. The intra-sequence temporal consistency loss is based on the idea of video temporal coherence [29,25,23]. While the previous works focus on learning the temporal order by considering individual frames, we use consistency as a tool for the learnt features to implicitly ignore background nuances and focus on the actual person attributes.…”

“…We propose using temporal coherence [29,25,23] as a form of self-supervision to maximally utilize the unlabeled data and learn discriminative person specific representations. Temporal coherence is motivated by the fact that features corresponding to a person in a tracklet should be focused on the discriminative aspects related to the person, such as clothing and gait, and ignore background nuances such as illumination and occlusion (see Fig.…”