Multiple Granularity Analysis for Fine-Grained Action Detection

Ni, Bingbing; Paramathayalan, Vignesh R.; Moulin, Pierre

doi:10.1109/cvpr.2014.102

Cited by 54 publications

(38 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Early work in activity recognition involved using hidden Markov models to learn latent action states [58], followed by discriminative SVM models that used key poses and action grammars [31,48,35]. Similar works have used hand-crafted features [40] or object-centric features [30] to recognize actions in fixed camera settings. More recent works have used dense trajectories [51] or deep learning features [19] to study actions.…”

Section: Related Workmentioning

confidence: 99%

Dense-Captioning Events in Videos

Krishna

Hata

Ren

et al. 2017

2017 IEEE International Conference on Computer Vision (ICCV)

1,051

950

View full text Add to dashboard Cite

Most natural videos contain numerous events. For example, in a video of a "man playing a piano", the video might also contain "another man dancing" or "a crowd clapping". We introduce the task of dense-captioning events, which involves both detecting and describing events in a video. We propose a new model that is able to identify all events in a single pass of the video while simultaneously describing the detected events with natural language. Our model introduces a variant of an existing proposal module that is designed to capture both short as well as long events that span minutes. To capture the dependencies between the events in a video, our model introduces a new captioning module that uses contextual information from past and future events to jointly describe all events. We also introduce ActivityNet Captions, a large-scale benchmark for dense-captioning events. ActivityNet Captions contains 20k videos amounting to 849 video hours with 100k total descriptions, each with it's unique start and end time. Finally, we report performances of our model for dense-captioning events, video retrieval and localization.

show abstract

Section: Related Workmentioning

confidence: 99%

Dense-Captioning Events in Videos

Krishna

Hata

Ren

et al. 2017

2017 IEEE International Conference on Computer Vision (ICCV)

1,051

950

View full text Add to dashboard Cite

show abstract

“…Most existing work on temporal action segmentation focuses on a fully supervised task [15,26,31,19,32,22,6,37,18,36,28]. Purely CNN based approaches such as structured segment networks [37] or temporal convolutional networks [18] have recently shown convincing results on several action segmentation benchmarks.…”

Section: Related Workmentioning

confidence: 99%

NeuralNetwork-Viterbi: A Framework for Weakly Supervised Video Learning

Richard

Kuehne

Iqbal

et al. 2018

2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition

125

187

View full text Add to dashboard Cite

Video learning is an important task in computer vision and has experienced increasing interest over the recent years. Since even a small amount of videos easily comprises several million frames, methods that do not rely on a frame-level annotation are of special importance. In this work, we propose a novel learning algorithm with a Viterbibased loss that allows for online and incremental learning of weakly annotated video data. We moreover show that explicit context and length modeling leads to huge improvements in video segmentation and labeling tasks and include these models into our framework. On several action segmentation benchmarks, we obtain an improvement of up to 10% compared to current state-of-the-art methods.

show abstract

“…In [10,22], human actions are modeled as space-time structures, using deformable part models [2]. In [15,18] discriminative handcentric features are explored for fine grained activity detection in cooking, i.e., relatively short sub-activities such as chop and fill. In [3], the detector is trained on CNN features extracted from the action tubes in space-time; however, evaluation is on relatively short video clips (i.e., several hundred frames) of relatively short actions.…”

Section: Related Workmentioning

confidence: 99%

Learning Activity Progression in LSTMs for Activity Detection and Early Detection

Sigal

Sclaroff

2016

2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

360

262

View full text Add to dashboard Cite

In this work we improve training of temporal deep models to better learn activity progression for activity detection and early detection tasks. Conventionally, when training a Recurrent Neural Network, specifically a Long Short Term Memory (LSTM) model, the training loss only considers classification error. However, we argue that the detection score of the correct activity category, or the detection score margin between the correct and incorrect categories, should be monotonically non-decreasing as the model observes more of the activity. We design novel ranking losses that directly penalize the model on violation of such monotonicities, which are used together with classification loss in training of LSTM models. Evaluation on ActivityNet shows significant benefits of the proposed ranking losses in both activity detection and early detection tasks.

show abstract

Multiple Granularity Analysis for Fine-Grained Action Detection

Cited by 54 publications

References 20 publications

Dense-Captioning Events in Videos

Dense-Captioning Events in Videos

NeuralNetwork-Viterbi: A Framework for Weakly Supervised Video Learning

Learning Activity Progression in LSTMs for Activity Detection and Early Detection

Contact Info

Product

Resources

About