Using closed captions to train activity recognizers that improve video retrieval

Gupta, Shreya; Mooney, Rohit J. Kate and Raymond J.

doi:10.1109/cvpr.2009.5204202

Cited by 4 publications

(5 citation statements)

References 17 publications

(27 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For increasingly large and complex datasets, manual labeling will become prohibitive. Automatic labeling using video subtitles [48] and movie scripts [20,26,73] is possible in some domains, but still requires manual verification. When using an incremental approach to image harvesting such as in [55], the initial set will largely affect the final variety of action performances.…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

A survey on vision-based human action recognition

Poppe

2010

Image and Vision Computing

2,024

979

View full text Add to dashboard Cite

Section: Discussionmentioning

confidence: 99%

“…Several automatic approaches have been proposed, for example using web image search results [55], video subtitles [48] and subtitle to movie script matching [20,26,73]. Gaidon et al [40] present an approach to rerank automatically extracted and aligned movie samples but manual verification is usually necessary.…”

Section: Obtaining and Labeling Training Datamentioning

confidence: 99%

A survey on vision-based human action recognition

Poppe

2010

Image and Vision Computing

2,024

979

View full text Add to dashboard Cite

“…They again used commentary as a "view" for action recognition with success. To train an activity classifier in an automated fashion, without the requirement of any manual labeling, Sonal and Mooney [7] make use of broadcast closed captions and used the system for video retrieval. There are a few works which focus on captioning in sports settings.…”

Section: Related Workmentioning

confidence: 99%

What and How Well You Performed? A Multitask Learning Approach to Action Quality Assessment

Parmar

Morris

2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

140

164

View full text Add to dashboard Cite

Can performance on the task of action quality assessment (AQA) be improved by exploiting a description of the action and its quality? Current AQA and skills assessment approaches propose to learn features that serve only one task -estimating the final score. In this paper, we propose to learn spatio-temporal features that explain three related tasks -fine-grained action recognition, commentary generation, and estimating the AQA score. A new multitask-AQA dataset, the largest to date, comprising of 1412 diving samples was collected to evaluate our approach (https: //github.com/ParitoshParmar/MTL-AQA). We show that our MTL approach outperforms STL approach using two different kinds of architectures: C3D-AVG and MSCADC. The C3D-AVG-MTL approach achieves the new state-of-the-art performance with a rank correlation of 90.44%. Detailed experiments were performed to show that MTL offers better generalization than STL, and representations from action recognition models are not sufficient for the AQA task and instead should be learned.Multitask AQA Action Quality Score:83.25/100

show abstract

“…22 However, the authors' approach only used a small dataset of seven videos and did not consider high-level natural language concept features in predicting sentiment or for aspect extraction. Besides this, caption mining was previously used in the context of scene segmentation, 23 video activity recognition, 24 and movie genre classification. 25 Targeting the automotive domain 26 demonstrated that emotional and factual knowledge, such as a sentiment of a product feature, can be extracted from text using a combination of lexical methods including SenticNet.…”

Section: Related Workmentioning

confidence: 99%