Revisiting Anchor Mechanisms for Temporal Action Localization

Yang, Le; Peng, Houwen; Zhang, Dingwen; Fu, Jianlong; Han, Junwei

doi:10.1109/tip.2020.3016486

Cited by 169 publications

(50 citation statements)

References 50 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For action localisation, we follow a two-stage paradigm: class-agnostic action proposal detection and proposal classification. To obtain high-quality action proposals, we first divide the entire video into equal-frame snippets; then use the CLIP image encoder with one Transformer layer to extract frame-wise embeddings for each snippet; and finally feed these embeddings to the off-the-shelf proposal detectors [37,74]. These detectors construct the feature pyramid, and make predictions in parallel, to determine actionness, centerness, and boundaries.…”

Section: Implementation Detailsmentioning

confidence: 99%

“…These detectors construct the feature pyramid, and make predictions in parallel, to determine actionness, centerness, and boundaries. Please refer to [37,74] for detailed detector architectures and optimisations. Note that, our method is flexible to the choice of proposal detectors, and we do not innovate on such candidate proposal procedures.…”

Section: Implementation Detailsmentioning

confidence: 99%

“…For action localisation, to obtain class-agnostic action proposals, we adopt the off-the-shelf proposal detectors [37,74]. Specifically, we first divide the entire video into equal-frame snippets; use the CLIP image encoder with one Transformer layer to extract frame-wise embeddings; feed these embeddings to the 6-layer feature pyramid; use 3 parallel prediction heads to determine the actionness, centerness, and boundaries respectively; finally, assemble all prediction results and use Soft-NMS [4] to suppress redundant proposals.…”

Section: B2 Action Localisationmentioning

confidence: 99%

See 2 more Smart Citations

Prompting Visual-Language Models for Efficient Video Understanding

Chen¹,

Han²,

Kunhao³

et al. 2021

Preprint

View full text Add to dashboard Cite

Visual-language pre-training has shown great success for learning joint visual-textual representations from largescale web data, demonstrating remarkable ability for "zeroshot" generalisation. This paper presents a simple method to efficiently adapt one pre-trained visual-language model to novel tasks with minimal training, and here, we consider video understanding tasks. Specifically, we propose to optimise a few random vectors, termed as "continuous prompt vectors", that convert the novel tasks into the same format as the pre-training objectives. In addition, to bridge the gap between static images and videos, temporal information is encoded with lightweight Transformers stacking on top of frame-wise visual features. Experimentally, we conduct extensive ablation studies to analyse the critical components and necessities. On 9 public benchmarks of action recognition, action localisation, and text-video retrieval, across closed-set, few-shot, open-set scenarios, we achieve competitive or state-of-the-art performance to existing methods, despite training significantly fewer parameters.

show abstract

Section: Implementation Detailsmentioning

confidence: 99%

Section: Implementation Detailsmentioning

confidence: 99%

Section: B2 Action Localisationmentioning

confidence: 99%

See 1 more Smart Citation

Prompting Visual-Language Models for Efficient Video Understanding

Chen¹,

Han²,

Kunhao³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…The recent success of convolutional neural networks (CNNs) in the video analysis [86] domain is commendable and has replaced statistical image processing, providing automated activity recognition [87,88], data prioritization [89], and many other useful tasks. The most effective methods for video surveillance are based on CNNs or their variants to classify abnormal actions/activities.…”

Section: Applying Fuzzy Logic: Why When and Where?mentioning

confidence: 99%

Fuzzy Logic in Surveillance Big Video Data Analysis

et al. 2021

View full text Add to dashboard Cite

CCTV cameras installed for continuous surveillance generate enormous amounts of data daily, forging the term Big Video Data (BVD). The active practice of BVD includes intelligent surveillance and activity recognition, among other challenging tasks. To efficiently address these tasks, the computer vision research community has provided monitoring systems, activity recognition methods, and many other computationally complex solutions for the purposeful usage of BVD. Unfortunately, the limited capabilities of these methods, higher computational complexity, and stringent installation requirements hinder their practical implementation in real-world scenarios, which still demand human operators sitting in front of cameras to monitor activities or make actionable decisions based on BVD. The usage of human-like logic, known as fuzzy logic, has been employed emerging for various data science applications such as control systems, image processing, decision making, routing, and advanced safety-critical systems. This is due to its ability to handle various sources of real-world domain and data uncertainties, generating easily adaptable and explainable data-based models. Fuzzy logic can be effectively used for surveillance as a complementary for huge-sized artificial intelligence models and tiresome training procedures. In this article, we draw researchers’ attention toward the usage of fuzzy logic for surveillance in the context of BVD. We carry out a comprehensive literature survey of methods for vision sensory data analytics that resort to fuzzy logic concepts. Our overview highlights the advantages, downsides, and challenges in existing video analysis methods based on fuzzy logic for surveillance applications. We enumerate and discuss the datasets used by these methods, and finally provide an outlook toward future research directions derived from our critical assessment of the efforts invested so far in this exciting field.

show abstract

“…While promising results have been obtained, a limitation is that they usually assume the video has been trimmed and aligned with text description. The earlier work for automatically tailoring videos is temporal action localization [14,31], which is to localize interesting actions in a video from a given set of actions. However, the predefined action set is usually small and unrealistic, which is far from actual need.…”

Section: Introductionmentioning

confidence: 99%

Local-enhanced Interaction for Temporal Moment Localization

Liang

Zhang

2021

Proceedings of the 2021 International Conference on Multimedia Retrieval

View full text Add to dashboard Cite

Temporal moment localization via language aims to localize a video span in an untrimmed video which best matches the given natural language query. In most previous works, they try to match the whole query feature with multiple moment proposals, or match a global video embedding with phrase or word level query features. However, these coarse interaction models will become insufficient when the query-video contains more complex relationship. To address this issue, we propose a multi-branches interaction model for temporal moment localization. Specifically, the query sentence and video are encoded into multiple feature embeddings over several semantic sub-spaces. Then, each phrase embedding filters on a video feature to generate an attention sequence, which is used to reweight the video features. Moreover, a dynamic pointer decoder is developed to iteratively regress the temporal boundary, which can prevent our model from falling into a local optimum. To validate the proposed method, we have conducted extensive experiments on two popular benchmark datasets Charade-STA and TACoS. The experimental performance surpasses other state-of-the-arts methods, which demonstrates the effectiveness of our proposed model. CCS CONCEPTS• Information systems → Novelty in information retrieval.

show abstract

Revisiting Anchor Mechanisms for Temporal Action Localization

Cited by 169 publications

References 50 publications

Prompting Visual-Language Models for Efficient Video Understanding

Prompting Visual-Language Models for Efficient Video Understanding

Fuzzy Logic in Surveillance Big Video Data Analysis

Local-enhanced Interaction for Temporal Moment Localization

Contact Info

Product

Resources

About