Action Recognition From Single Timestamp Supervision in Untrimmed Videos

Moltisanti, Davide; Fidler, Sanja; Damen, Dima

doi:10.1109/cvpr.2019.01015

Cited by 59 publications

(49 citation statements)

References 31 publications

(58 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…from [31]. This plateau function has three parameters and it is differentiable with respect to them: λ c controls the center of the plateau, λ w is the width and λ s is the sharpness of the plateau function.…”

Section: Approximate Differentiable Energy E *mentioning

confidence: 99%

FIFA: Fast Inference Approximation for Action Segmentation

Souri¹,

Farha²,

Despinoy³

et al. 2021

Preprint

View full text Add to dashboard Cite

We introduce FIFA, a fast approximate inference method for action segmentation and alignment. Unlike previous approaches, FIFA does not rely on expensive dynamic programming for inference. Instead, it uses an approximate differentiable energy function that can be minimized using gradient-descent. FIFA is a general approach that can replace exact inference improving its speed by more than 5 times while maintaining its performance. FIFA is an anytime inference algorithm that provides a better speed vs. accuracy trade-off compared to exact inference. We apply FIFA on top of state-of-the-art approaches for weakly supervised action segmentation and alignment as well as fully supervised action segmentation. FIFA achieves state-of-the-art results on most metrics on two action segmentation datasets.

show abstract

Section: Approximate Differentiable Energy E *mentioning

confidence: 99%

FIFA: Fast Inference Approximation for Action Segmentation

Souri¹,

Farha²,

Despinoy³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…At present, a large number of researchers have been working in the field of video-based human action recognition, and have achieved many surprising results, 1–6 making the current human action recognition with very high recognition accuracy. Video-based human action recognition is mainly developed on the understanding of color video.…”

Section: Related Workmentioning

confidence: 99%

“…Video-based human action recognition is a very popular direction in the field of computer vision, 1–6 because it has been widely used in people's lives, such as video surveillance, human-computer interaction, video understand -ing, smart medicine and so on.In particular, the launch of the Kinect sensor camera by Microsoft has triggered a surge in the use of human skeleton information to analyze human action. Because of its powerful skeleton tracking function, Kinect enables researchers to analyze human action using human skeleton motion trajectories and has been very successful in many fields.…”

Section: Introductionmentioning

confidence: 99%

A novel algorithm of fast CPR quality evaluation based on kinect

Xie

Luo

Jie

et al. 2020

Journal of Algorithms & Computational Technology

View full text Add to dashboard Cite

Human action recognition is a very popular field in computer vision research, and the research results are widely used in people's lives. This paper explores Kinect-based algorithm of human action recognition and applies it to the quality evaluation of cardiopulmonary resuscitation (CPR) operation. At present, the main means of CRP training is through physical auxiliary equipment, which has a large limitation and can only be carried out under specific conditions. CPR simulation training under general conditions can be effectively carried out by means of computer vision, which is a strategy worth popularizing. Using Kinect's powerful skeleton tracking capabilities to obtain key human skeleton data and then perform fine-grained human action analysis. Our model can obtain the critical compression depth (CCD) and compression frequency (CCF) of CPR. Compared with the-state-of-the-art, our algorithm has better stability and real-time performance. At the same time, our algorithm improves the time efficiency by about 60% while guaranteeing high accuracy. In addition, we guide the human body to perform standard movements by setting joint angle specifications.Moreover, our system has been proven to be valid by professional medical staff.

show abstract

“…While transcript-level and set-level supervision significantly reduce the annotation effort, the performance is not satisfying and there is still a gap compared to fully supervised approaches. In this paper, inspired by the recently introduced timestamp supervision for action recognition [31], we propose to use timestamp supervision for the action segmentation task to address the limitations of the current weakly supervised approaches. For timestamp supervision, only one frame is annotated from each action segment as illustrated in Fig.…”

Section: Introductionmentioning

confidence: 99%

Temporal Action Segmentation from Timestamp Supervision

Li¹,

Farha²,

Gall³

2021

Preprint

View full text Add to dashboard Cite

Temporal action segmentation approaches have been very successful recently. However, annotating videos with frame-wise labels to train such models is very expensive and time consuming. While weakly supervised methods trained using only ordered action lists require less annotation effort, the performance is still worse than fully supervised approaches. In this paper, we propose to use timestamp supervision for the temporal action segmentation task. Timestamps require a comparable annotation effort to weakly supervised approaches, and yet provide a more supervisory signal. To demonstrate the effectiveness of timestamp supervision, we propose an approach to train a segmentation model using only timestamps annotations. Our approach uses the model output and the annotated timestamps to generate frame-wise labels by detecting the action changes. We further introduce a confidence loss that forces the predicted probabilities to monotonically decrease as the distance to the timestamps increases. This ensures that all and not only the most distinctive frames of an action are learned during training. The evaluation on four datasets shows that models trained with timestamps annotations achieve comparable performance to the fully supervised approaches.

show abstract

Action Recognition From Single Timestamp Supervision in Untrimmed Videos

Cited by 59 publications

References 31 publications

FIFA: Fast Inference Approximation for Action Segmentation

FIFA: Fast Inference Approximation for Action Segmentation

A novel algorithm of fast CPR quality evaluation based on kinect

Temporal Action Segmentation from Timestamp Supervision

Contact Info

Product

Resources

About