Weakly Supervised Action Detection

Siva, Parthipan; Xiang, Tao

doi:10.5244/c.25.65

Cited by 56 publications

(64 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Traditionally, action localization or detection is performed by sliding window based approaches [20,11,3,27]. For instance, Siva et al [20] proposed a supervised model based on multiple-instance-learning to slide over subvolumes both spatially and temporally for action detection.…”

Section: Related Workmentioning

confidence: 99%

“…For instance, Siva et al [20] proposed a supervised model based on multiple-instance-learning to slide over subvolumes both spatially and temporally for action detection. Instead of performing an exhaustive search through sliding over the whole video volumes, Oneata et al [13] put forward a branch-and-bound search approach to achieve the time-efficiency.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Searching Action Proposals via Spatial Actionness Estimation and Temporal Path Inference and Tracking

Li¹,

Ying³

et al. 2017

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Abstract. In this paper, we address the problem of searching action proposals in unconstrained video clips. Our approach starts from actionness estimation on frame-level bounding boxes, and then aggregates the bounding boxes belonging to the same actor across frames via linking, associating, tracking to generate spatial-temporal continuous action paths. To achieve the target, a novel actionness estimation method is firstly proposed by utilizing both human appearance and motion cues. Then, the association of the action paths is formulated as a maximum set coverage problem with the results of actionness estimation as a priori. To further promote the performance, we design an improved optimization objective for the problem and provide a greedy search algorithm to solve it. Finally, a tracking-by-detection scheme is designed to further refine the searched action paths. Extensive experiments on two challenging datasets, UCF-Sports and UCF-101, show that the proposed approach advances state-of-the-art proposal generation performance in terms of both accuracy and proposal quantity.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Searching Action Proposals via Spatial Actionness Estimation and Temporal Path Inference and Tracking

Li¹,

Ying³

et al. 2017

Lecture Notes in Computer Science

View full text Add to dashboard Cite

show abstract

“…The results also suggest that it outperforms the original MIML model [33], the state-ofthe-art weakly supervised approaches [25,24,18], as well as fully supervised methods [4,29,22,32] in the literature across the three datasets. The paper is organised as follows: Section 2 provides a review of related work for MIML techniques and weakly supervised action detection; Section 3 details the feature representation of video under the weakly supervised setting; Section 4 formulates the proposed framework and introduces the generation of instances and bags in the setting of weakly supervised action detection; Section 5 describes the experiments, such as data and implementation details; Section 6 demonstrates results and analysis of the experiments; 3 finally Section 7 concludes this work and points out possible future work.…”

Section: Introductionmentioning

confidence: 73%

“…Therefore, it cannot be directly applied to weakly labelled video data that provides the presence of multiple activities without any spatio-temporal localisation in each video. A multi-instance learning approach that optimises intraclass and interclass distances for action annotation and detection is introduced in [25]. The training does not require the manual annotation of rough locations of actions, but it still has the same assumption of one true instance of one of the action classes per video.…”

Section: Related Workmentioning

confidence: 99%

“…It is then extended and improved for weakly supervised annotation in [24], by focusing on negative mining in multi-instance learning. Neither of the approaches [25,24] however explicitly models the interclass relationships within each video (or bag), and thus may struggle in cases where multiple action classes are simultaneously presented in a video (i.e. multi-label learning).…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Weakly supervised activity analysis with spatio-temporal localisation

Sridhar

Cohn

et al. 2016

Neurocomputing

View full text Add to dashboard Cite

In computer vision, an increasing number of weakly annotated videos have become available, due to the fact it is often difficult and time consuming to annotate all the details in the videos collected. Learning methods that analyse human activities in weakly annotated video data have gained great interest in recent years. They are categorised as "weakly supervised learning", and usually form a multi-instance multi-label (MIML) learning problem. In addition to the commonly known difficulties of MIML learning, i.e. ambiguities in instances and labels, a weakly supervised method also has to cope with large data size, high dimensionality, and a large proportion of noisy examples usually found in video data. In this work, we propose a novel learning framework that iteratively optimises over a scalable MIML model and an instance selection process incorporating pairwise spatio-temporal smoothing during training. Such learned knowledge is then generalised to testing via a noise removal process based on the support vector data description algorithm. According to the experiments on three challenging benchmark video datasets, the proposed framework yields a more discriminative MIML model and less noisy training and testing data, and thus improves the system performance. It outperforms the state-of-the-art weakly supervised and even fully supervised approaches in the literature, in terms of annotating and detecting actions of a single person and interactions between a pair of people.

show abstract

W-TALC: Weakly-Supervised Temporal Activity Localization and Classification

Shrivastava

Roy

Roy-Chowdhury

2018

Lecture Notes in Computer Science

291

335

View full text Add to dashboard Cite

Most activity localization methods in the literature suffer from the burden of frame-wise annotation requirement. Learning from weak labels may be a potential solution towards reducing such manual labeling effort. Recent years have witnessed a substantial influx of tagged videos on the Internet, which can serve as a rich source of weakly-supervised training data. Specifically, the correlations between videos with similar tags can be utilized to temporally localize the activities. Towards this goal, we present W-TALC, a Weakly-supervised Temporal Activity Localization and Classification framework using only video-level labels. The proposed network can be divided into two sub-networks, namely the Two-Stream based feature extractor network and a weakly-supervised module, which we learn by optimizing two complimentary loss functions. Qualitative and quantitative results on two challenging datasets -Thumos14 and ActivityNet1.2, demonstrate that the proposed method is able to detect activities at a fine granularity and achieve better performance than current state-of-the-art methods. Codes available at https://github.com/sujoyp/wtalc-pytorch

show abstract

Weakly Supervised Action Detection

Cited by 56 publications

References 16 publications

Searching Action Proposals via Spatial Actionness Estimation and Temporal Path Inference and Tracking

Searching Action Proposals via Spatial Actionness Estimation and Temporal Path Inference and Tracking

Weakly supervised activity analysis with spatio-temporal localisation

W-TALC: Weakly-Supervised Temporal Activity Localization and Classification

Contact Info

Product

Resources

About