CTAP: Complementary Temporal Action Proposal Generation

Gao, Jiyang; Chen, Kan; Nevatia, Ram

doi:10.1007/978-3-030-01216-8_5

Cited by 181 publications

(144 citation statements)

References 26 publications

Supporting

Mentioning

142

Contrasting

Order By: Relevance

“…To achieve high prediction accuracy, most of the existing state-of-the-art algorithms for temporal action proposals use supervised deep learning approaches [3,14,15,23]. Such approaches require large amount of labeled videos.…”

Section: Introductionmentioning

confidence: 99%

Learning Temporal Action Proposals With Fewer Labels

Cao

Niebles

2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

View full text Add to dashboard Cite

Temporal action proposals are a common module in action detection pipelines today. Most current methods for training action proposal modules rely on fully supervised approaches that require large amounts of annotated temporal action intervals in long video sequences. The large cost and effort in annotation that this entails motivate us to study the problem of training proposal modules with less supervision. In this work, we propose a semi-supervised learning algorithm specifically designed for training temporal action proposal networks. When only a small number of labels are available, our semi-supervised method generates significantly better proposals than the fully-supervised counterpart and other strong semi-supervised baselines. We validate our method on two challenging action detection video datasets, ActivityNet v1.3 and THUMOS14. We show that our semi-supervised approach consistently matches or outperforms the fully supervised state-of-the-art approaches.

show abstract

Section: Introductionmentioning

confidence: 99%

Learning Temporal Action Proposals With Fewer Labels

Cao

Niebles

2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

View full text Add to dashboard Cite

show abstract

“…Performance comparison. For the THUMOS-14 dataset, we plot the AR vs. AN and Recall@AN=100 vs. tIoU curves of SRG and 6 previous methods, S-CNN-prop [14], TAG [18], TURN [12], CTAP [21], BSN [22], and MGG [23], in Fig. 8, where SRG outperformed state-of-the-art proposal generation methods on both AR vs. AN and Recall@AN=100 vs. tIoU curves.…”

Section: Methodsmentioning

confidence: 99%

“…Among them, 200 and 212 videos in the validation and test sets, respectively, have temporal annotations with 20 action classes. We trained our models on the validation set and conducted an evaluation on the test set, as done in previous works [12], [15], [21], [22], [23].…”

Section: Experiments a Datasetsmentioning

confidence: 99%

“…For both datasets, we set τ to 0.1 with an increasing step size of 0.1 to generate temporal intervals based on the relatedness and weighted relatedness score sequences. Note that, unlike TAG [18], CTAP [21], and MGG [23] using actionness scores, we do not group multiple temporal intervals since our score sequences consider only the action instance related to the…”

Section: B Temporal Action Proposal Generationmentioning

confidence: 99%

See 1 more Smart Citation

SRG: Snippet Relatedness-Based Temporal Action Proposal Generator

Eun

Lee

Moon

et al. 2020

IEEE Trans. Circuits Syst. Video Technol.

View full text Add to dashboard Cite

Recent temporal action proposal generation approaches have suggested integrating segment-and snippet scorebased methodologies to produce proposals with high recall and accurate boundaries. In this paper, different from such a hybrid strategy, we focus on the potential of the snippet score-based approach. Specifically, we propose a new snippet score-based method, named Snippet Relatedness-based Generator (SRG), with a novel concept of "snippet relatedness". Snippet relatedness represents which snippets are related to a specific action instance. To effectively learn this snippet relatedness, we present "pyramid non-local operations" for locally and globally capturing long-range dependencies among snippets. By employing these components, SRG first produces a 2D relatedness score map that enables the generation of various temporal intervals reliably covering most action instances with high overlap. Then, SRG evaluates the action confidence scores of these temporal intervals and refines their boundaries to obtain temporal action proposals. On THUMOS-14 and ActivityNet-1.3 datasets, SRG outperforms state-of-the-art methods for temporal action proposal generation. Furthermore, compared to competing proposal generators, SRG leads to significant improvements in temporal action detection.

show abstract

“…Moreover, used metrics cannot be said to be of an online nature. In other words, metrics such as the mean Average Precision (mAP) [15] or the Calibrated Previous evaluation protocols for Online Action Detection (OAD) were based on: 1) running the online methods through all videos; 2) applying the offline metric on the obtained results. Additionally, offline metrics proposed so far do not consider the background in their evaluation.…”

Section: Introductionmentioning

confidence: 99%

Rethinking Online Action Detection in Untrimmed Videos: A Novel Online Evaluation Protocol

et al. 2020

View full text Add to dashboard Cite

The Online Action Detection (OAD) problem needs to be revisited. Unlike traditional offline action detection approaches, where the evaluation metrics are clear and well established, in the OAD setting we find very few works and no consensus on the evaluation protocols to be used. In this work we propose to rethink the OAD scenario, clearly defining the problem itself and the main characteristics that the models which are considered online must comply with. We also introduce a novel metric: the Instantaneous Accuracy (IA). This new metric exhibits an online nature and solves most of the limitations of the previous metrics. We conduct a thorough experimental evaluation on 3 challenging datasets, where the performance of various baseline methods is compared to that of the state-of-the-art. Our results confirm the problems of the previous evaluation protocols, and suggest that an IA-based protocol is more adequate to the online scenario. The baselines models and a development kit with the novel evaluation protocol are publicly available here (https://github.com/gramuah/ia).

show abstract

CTAP: Complementary Temporal Action Proposal Generation

Cited by 181 publications

References 26 publications

Learning Temporal Action Proposals With Fewer Labels

Learning Temporal Action Proposals With Fewer Labels

SRG: Snippet Relatedness-Based Temporal Action Proposal Generator

Rethinking Online Action Detection in Untrimmed Videos: A Novel Online Evaluation Protocol

Contact Info

Product

Resources

About