Self-Supervised Learning for Semi-Supervised Temporal Action Proposal

Wang, Xiang; Zhang, Shiwei; Qing, Zhiwu; Shao, Yuanjie; Gao, Changxin; Sang, Nong

doi:10.48550/arxiv.2104.03214

Cited by 4 publications

(7 citation statements)

References 47 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Channel-Separated Convolutional Network (CSN) [13] aims to reduce the parameters of 3D convolution, and extract useful information by finding important channels simultaneously. It can efficiently learn feature representation Note that this module is borrowed from our SSTAP [18].…”

Section: Channel-separated Convolutional Networkmentioning

confidence: 99%

“…Temporal shift operation for action recognition is first applied in TSM [9], and then applied as a kind of perturbations in SSTAP [18] for semi-supervised learning. Here we reuse the perturbation as the feature augmentation.…”

Section: Data Augmentation Modulementioning

confidence: 99%

“…Action understanding is an important area in computer vision, and it draws growing attentions from both industry and academia because its use in human computer interaction, public security and some other far reaching applications. It includes many sub-research directions, such as Action Recognition [15,6,7], Temporal Action Detection [10,11,18], Spatio-Temporal Action Detection [12,8], etc. In this report, we introduce our method for the temporal action detection task in the 6-th ActivityNet challenge [2].…”

Section: Introductionmentioning

confidence: 99%

“…Figure 2. Detailed diagram of the data augmentation module.Note that this module is borrowed from our SSTAP[18].…”

mentioning

confidence: 99%

See 3 more Smart Citations

Proposal Relation Network for Temporal Action Detection

Wang¹,

Qing²,

Huang³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

This technical report presents our solution for temporal action detection task in AcitivityNet Challenge 2021. The purpose of this task is to locate and identify actions of interest in long untrimmed videos. The crucial challenge of the task comes from that the temporal duration of action varies dramatically, and the target actions are typically embedded in a background of irrelevant activities. Our solution builds on BMN [10], and mainly contains three steps: 1) action classification and feature encoding by Slowfast [6], CSN [13] and ViViT [1]; 2) proposal generation. We improve BMN by embedding the proposed Proposal Relation Network (PRN), by which we can generate proposals of high quality; 3) action detection. We calculate the detection results by assigning the proposals with corresponding classification results. Finally, we ensemble the results under different settings and achieve 44.7% on the test set, which improves the champion result in ActivityNet 2020 [17] by 1.9% in terms of average mAP.

show abstract

Section: Channel-separated Convolutional Networkmentioning

confidence: 99%

Section: Data Augmentation Modulementioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

“…Figure 2. Detailed diagram of the data augmentation module.Note that this module is borrowed from our SSTAP[18].…”

mentioning

confidence: 99%

See 2 more Smart Citations

Proposal Relation Network for Temporal Action Detection

Wang¹,

Qing²,

Huang³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Following mainstream action proposal generation methods [16,17,15,25,10,5,2,26,19,20,23,24], we preextract features for each video. Specifically, for a video which contains l frames, the whole video can be divided into N clips uniformly.…”

Section: Extracting Featuresmentioning

confidence: 99%

Exploring Stronger Feature for Temporal Action Localization

Qing,

Wang,

Huang

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Temporal action localization aims to localize starting and ending time with action category. Limited by GPU memory, mainstream methods pre-extract features for each video. Therefore, feature quality determines the upper bound of detection performance. In this technical report, we explored classic convolution-based backbones and the recent surge of transformer-based backbones. We found that the transformer-based methods can achieve better classification performance than convolution-based, but they cannot generate accuracy action proposals. In addition, extracting features with larger frame resolution to reduce the loss of spatial information can also effectively improve the performance of temporal action localization. Finally, we achieve 42.42% in terms of mAP on validation set with a single SlowFast [9] feature by a simple combination: BMN [16]+TCANet [19], which is 1.87% higher than the result of 2020 [20]'s multi-model ensemble. Finally, we achieve Rank 1st on the CVPR2021 HACS supervised Temporal Action Localization Challenge.

show abstract