Boundary Content Graph Neural Network for Temporal Action Proposal Generation

Bai, Yueran; Wang, Yingying; Tong, Yunhai; Yang, Yang; Liu, Qiyue; Liu, Junhui

doi:10.1007/978-3-030-58604-1_8

Cited by 117 publications

(53 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Comparative analysis. We compare our work to all seminal efforts that evaluate on these two standard benchmarks (Zhao et al, 2017;Chao et al, 2018; Zeng et al, 2019;Xu et al, 2020;Bai et al, 2020;Chen et al, 2020;Su et al, 2021). As in previous efforts (Zeng et al, 2019;Chen et al, 2020;Xu et al, 2020), we perform an additional test when combining our work with the additional power of proposal-to-proposal relations from PGCN (Zeng et al, 2019) and the temporal aggregation from MUSES (Liu et al, 2021).…”

Section: Methodsmentioning

confidence: 99%

“…Instead, temporal action localization works aim to locate actions in untrimmed videos as well as classify them. Most works, like ours, investigate proposal generation and proposal evaluation (Zhao et al, 2017;Chao et al, 2018;Lin et al, 2018;Lin et al, 2019;Liu et al, 2019;Long et al, 2019;Xu et al, 2020;Bai et al, 2020;Chen et al, 2020;Su et al, 2021), but some just focus on proposal evaluation, such as (Zeng et al, 2019;Liu et al, 2021).…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

TVNet: Temporal Voting Network for Action Localization

Wang¹,

Damen²,

Mirmehdi³

et al. 2022

Preprint

View full text Add to dashboard Cite

We propose a Temporal Voting Network (TVNet) for action localization in untrimmed videos. This incorporates a novel Voting Evidence Module to locate temporal boundaries, more accurately, where temporal contextual evidence is accumulated to predict frame-level probabilities of start and end action boundaries. Our action-independent evidence module is incorporated within a pipeline to calculate confidence scores and action classes. We achieve an average mAP of 34.6% on ActivityNet-1.3, particularly outperforming previous methods with the highest IoU of 0.95. TVNet also achieves mAP of 56.0% when combined with PGCN and 59.1% with MUSES at 0.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

TVNet: Temporal Voting Network for Action Localization

Wang¹,

Damen²,

Mirmehdi³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…MGG [32] combines anchor-based methods and boundarybased methods to generate proposals. The works in [52,1] propose to use graph convolutional networks [22] to model temporal relationships in the input video. BMN [29] proposes a boundary-matching mechanism for the confidence evaluation of densely distributed proposals in an end-to-end pipeline.…”

Section: Related Workmentioning

confidence: 99%

“…The evaluation metric of temporal action detection is mAP, which calculates the Average Precision under multiple IoU thresholds for each action cate- We adopt the two-stage "detection by classifying proposals" temporal action detection framework to combine our proposals with action classifiers. For fair comparisons, following [31,29,52,1], on ActivityNet v1.3, we adopt top-1 video-level classification results generated by method [55] and use confidence scores of BMN proposals for detection results retrieving. On THUMOS14, following BMN [29], we also use both top-2 video-level classification results generated by UntrimmedNet [45].…”

Section: Action Detection With Our Proposalsmentioning

confidence: 99%

Self-Supervised Learning for Semi-Supervised Temporal Action Proposal

Wang¹,

Zhang²,

Qing³

et al. 2021

Preprint

View full text Add to dashboard Cite

Self-supervised learning presents a remarkable performance to utilize unlabeled data for various video tasks. In this paper, we focus on applying the power of selfsupervised methods to improve semi-supervised action proposal generation. Particularly, we design an effective Selfsupervised Semi-supervised Temporal Action Proposal (SSTAP) framework. The SSTAP contains two crucial branches, i.e., temporal-aware semi-supervised branch and relation-aware self-supervised branch. The semisupervised branch improves the proposal model by introducing two temporal perturbations, i.e., temporal feature shift and temporal feature flip, in the mean teacher framework. The self-supervised branch defines two pretext tasks, including masked feature reconstruction and clip-order prediction, to learn the relation of temporal clues. By this means, SSTAP can better explore unlabeled videos, and improve the discriminative abilities of learned action features. We extensively evaluate the proposed SSTAP on THUMOS14 and ActivityNet v1.3 datasets. The experimental results demonstrate that SSTAP significantly outperforms state-of-the-art semi-supervised methods and even matches fully-supervised methods. Code is available at https://github.com/wangxiang1230/SSTAP.

show abstract

“…To demonstrate the effectiveness of the proposed MDN, we compare it with more than 10 state-of-the-art temporal action detection algorithms, inluding DBG (AAAI'20) [21], G-TAD (CVPR'20) [37], BC-GNN (ECCV'20) [1], BU-TAL (ECCV'20) [39], TSI (ACCV'20) [25], etc. For a fair comparison, we use the same video feature representation and post-processing step.…”

Section: Comparisons With the State-of-the-artsmentioning

confidence: 99%

Multi-scale Dynamic Network for Temporal Action Detection

Ren

Shen

et al. 2021

Proceedings of the 2021 International Conference on Multimedia Retrieval

Self Cite

View full text Add to dashboard Cite

In recent years, as the fundamental task in video understanding, Temporal Action Detection is attracting extensive attention. Most existing approaches use the same model parameters to process all input videos, which are not adaptive to the input video during the inference stage. In this paper, we propose a novel model termed Multi-scale Dynamic Network (MDN) to tackle this problem. The proposed MDN model incorporates multiple Multi-scale Dynamic Modules (MDMs). Each MDM can generate video-specific and segment-specific convolution kernels based on video content from different scales and adaptively capture rich semantic information for the prediction. Besides, we also design a new Edge Suppression Loss (ESL) function for MDN to pay more attention to hard examples. Extensive experiments conducted on two popular benchmarks ActivityNet-1.3 and THUMOS-14 show that the proposed MDN model achieves the state-of-the-art performance. CCS CONCEPTS• Computing methodologies → Activity recognition and understanding.

show abstract

Boundary Content Graph Neural Network for Temporal Action Proposal Generation

Cited by 117 publications

References 25 publications

TVNet: Temporal Voting Network for Action Localization

TVNet: Temporal Voting Network for Action Localization

Self-Supervised Learning for Semi-Supervised Temporal Action Proposal

Multi-scale Dynamic Network for Temporal Action Detection

Contact Info

Product

Resources

About