BSN: Boundary Sensitive Network for Temporal Action Proposal Generation

Lin, Tianwei; Zhao, Xu; Su, Haisheng; Wang, Chongjing; Yang, Ming

doi:10.48550/arxiv.1806.02964

Cited by 26 publications

(44 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, the performance of the employed methods was limited by the hand-crafted feature representations. Recently, deep networks were applied to action localization to achieve improved performance [28], [29], [30], [31], [13], [32]. In S-CNN [28], multi-stage CNN that utilizes 3D-ConvNet (C3D) [9] is used to simultaneously capture the spatial and temporal features.…”

Section: B Temporal Action Localizationmentioning

confidence: 99%

Temporal Action Localization using Long Short-Term Dependency

Zhou¹,

Li²,

Kung³

2019

Preprint

View full text Add to dashboard Cite

Temporal action localization in untrimmed videos is an important but difficult task. Difficulties are encountered in the application of existing methods when modeling temporal structures of videos. In the present study, we developed a novel method, referred to as Gemini Network, for effective modeling of temporal structures and achieving high-performance temporal action localization. The significant improvements afforded by the proposed method are attributable to three major factors. First, the developed network utilizes two subnets for effective modeling of temporal structures. Second, three parallel feature extraction pipelines are used to prevent interference between the extractions of different stage features. Third, the proposed method utilizes auxiliary supervision, with the auxiliary classifier losses affording additional constraints for improving the modeling capability of the network. As a demonstration of its effectiveness, the Gemini Network was used to achieve state-of-the-art temporal action localization performance on two challenging datasets, namely, THUMOS14 and ActivityNet.

show abstract

Section: B Temporal Action Localizationmentioning

confidence: 99%

Temporal Action Localization using Long Short-Term Dependency

Zhou¹,

Li²,

Kung³

2019

Preprint

View full text Add to dashboard Cite

show abstract

“…Improved-BSN. BSN [7] also generates the proposals in a bottom-up fashion which first evaluates the probabilities of each temporal location being in the starting, ending and middling regions. Then through combining the high probability boundary locations, it can generate abundant proposals with flexible durations and confidence scores.…”

Section: Multi-granularity Fusion Networkmentioning

confidence: 99%

“…However, the pro-posals generated in a top-down fashion are doomed to have imprecise boundaries though with regression. Under this circumstance, the other type of methods [7,11] have drawn much attention in the community recently which tackle this problem in a bottom-up fashion, where the input video is evaluated in a finer-level. [7] is a typical method in this type which proposes the Boundary Sensitive Network (BSN) to generate proposals with flexible durations and reliable confidence scores.…”

Section: Task Introductionmentioning

confidence: 99%

“…Under this circumstance, the other type of methods [7,11] have drawn much attention in the community recently which tackle this problem in a bottom-up fashion, where the input video is evaluated in a finer-level. [7] is a typical method in this type which proposes the Boundary Sensitive Network (BSN) to generate proposals with flexible durations and reliable confidence scores. Though BSN achieves convincing performance in this manner, it still suffers from many drawbacks.…”

Section: Task Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Multi-Granularity Fusion Network for Proposal and Activity Localization: Submission to ActivityNet Challenge 2019 Task 1 and Task 2

Su,

Zhao,

Liu

2019

Preprint

Self Cite

View full text Add to dashboard Cite

This technical report presents an overview of our solution used in the submission to ActivityNet Challenge 2019 Task 1 (temporal action proposal generation) and Task 2 (temporal action localization/detection). Temporal action proposal indicates the temporal intervals containing the actions and plays an important role in temporal action localization. Top-down and bottom-up methods are the two main categories used for proposal generation in the existing literature. In this paper, we devise a novel Multi-Granularity Fusion Network (MGFN) to combine the proposals generated from different frameworks for complementary filtering and confidence re-ranking. Specifically, we consider the diversity comprehensively from multiple perspectives, e.g. the characteristic aspect, the data aspect, the model aspect and the result aspect. Our MGFN achieves the state-of-theart performance on the temporal action proposal task with 69.85 AUC score and the temporal action localization task with 38.90 mAP on the challenge testing set.

show abstract

“…It leads to an important yet challenging task for video analysis: Temporal Action Localization (TAL), which requires to not only classify the untrimmed videos into specific categories accurately, but also locate the temporal boundaries of action instances precisely. Although substantial progress has been achieved on this task [41], [26], [39], [16], [6], [18], [10], [9], it is still limited for industrial applications due to the huge amount of temporal annotations used for training such a deep learning based model in a fully-supervised manner, which are laborintensive to annotate especially for a large-scale dataset. On the contrary, weak labels such as video-level labels are much easier to obtain, hence many current works try to handle this problem under weak supervision.…”

Section: Introductionmentioning

confidence: 99%

Transferable Knowledge-Based Multi-Granularity Aggregation Network for Temporal Action Localization: Submission to ActivityNet Challenge 2021

Su¹,

Zhuang²,

Li³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

This technical report presents an overview of our solution used in the submission to 2021 HACS Temporal Action Localization Challenge on both Supervised Learning Track and Weakly-Supervised Learning Track. Temporal Action Localization (TAL) requires to not only precisely locate the temporal boundaries of action instances, but also accurately classify the untrimmed videos into specific categories. However, Weakly-Supervised TAL indicates locating the action instances using only video-level class labels. In this paper, to train a supervised temporal action localizer, we adopt Temporal Context Aggregation Network (TCANet) to generate high-quality action proposals through "local and global" temporal context aggregation and complementary as well as progressive boundary refinement. As for the WSTAL, a novel framework is proposed to handle the poor quality of CAS generated by simple classification network, which can only focus on local discriminative parts, rather than locate the entire interval of target actions. Specifically, we propose to utilize convolutional kernels with varied dilation rates to enlarge the receptive fields, which is found to be capable of transferring the discriminative information to surrounding non-discriminative regions. Then we design a cascaded module with proposed Online Adversarial Erasing (OAE) mechanism to further mine more relevant regions of target actions through feeding the erased feature maps of discovered regions back to the system. Besides, inspired by the transfer learning method, we also adopt an additional module to transfer the knowledge from trimmed videos (HACS Clips dataset) to untrimmed videos (HACS Segments dataset), aiming at promoting the classification performance on untrimmed videos. Finally, we employ a boundary regression module embedded with Outer-Inner-Contrastive (OIC) loss to automatically predict the bound- * Corresponding author.aries based on the enhanced CAS. Our proposed scheme achieves 39.91 and 29.78 average mAP on the challenge testing set of supervised and weakly-supervised temporal action localization track respectively.

show abstract

BSN: Boundary Sensitive Network for Temporal Action Proposal Generation

Cited by 26 publications

References 26 publications

Temporal Action Localization using Long Short-Term Dependency

Temporal Action Localization using Long Short-Term Dependency

Multi-Granularity Fusion Network for Proposal and Activity Localization: Submission to ActivityNet Challenge 2019 Task 1 and Task 2

Transferable Knowledge-Based Multi-Granularity Aggregation Network for Temporal Action Localization: Submission to ActivityNet Challenge 2021

Contact Info

Product

Resources

About