Key frame extraction for salient activity recognition

Kulhare, Sourabh; Sah, Shagan; Pillai, Suhas; Ptucha, Raymond

doi:10.1109/icpr.2016.7899739

Cited by 25 publications

(14 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Conventional Methods: Many earlier approaches in this domain rely on using a segmentation based pipline. Such methods typically extract optical flow and SIFT features [9,[37][38][39]. One of the first works [9,37] describe a video with optical flow and detect local minimum changes in terms of similarity between successive frames.…”

Section: Key Frame Detectionmentioning

confidence: 99%

“…Such methods typically extract optical flow and SIFT features [9,[37][38][39]. One of the first works [9,37] describe a video with optical flow and detect local minimum changes in terms of similarity between successive frames. Later works improved upon this pipline by using keypoints detection for feature extraction [38,39], extracting local features via a SIFT descriptor and pooling the keypoints to find key frames in videos.…”

Section: Key Frame Detectionmentioning

confidence: 99%

“…We assume there are training videos without annotated key frames, and our goal is to train a deep neural network that can automatically annotate key frames in training videos. Efficient key frame extraction modules have also enabled a boost in performance of other high-level visual tasks, such as video summarization [1][2][3][4][5][6][7][8], action recognition [9][10][11], video retrieval [12], visual simultaneous localization and mapping (SLAM) [13][14][15], video annotation [16] and realistic videos generation [17]. For human action recognition, the key frames are the ones that can well represent the whole action.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Self-Supervised Learning to Detect Key Frames in Videos

Gilani

Feng

Zhang

et al. 2020

Sensors

View full text Add to dashboard Cite

Detecting key frames in videos is a common problem in many applications such as video classification, action recognition and video summarization. These tasks can be performed more efficiently using only a handful of key frames rather than the full video. Existing key frame detection approaches are mostly designed for supervised learning and require manual labelling of key frames in a large corpus of training data to train the models. Labelling requires human annotators from different backgrounds to annotate key frames in videos which is not only expensive and time consuming but also prone to subjective errors and inconsistencies between the labelers. To overcome these problems, we propose an automatic self-supervised method for detecting key frames in a video. Our method comprises a two-stream ConvNet and a novel automatic annotation architecture able to reliably annotate key frames in a video for self-supervised learning of the ConvNet. The proposed ConvNet learns deep appearance and motion features to detect frames that are unique. The trained network is then able to detect key frames in test videos. Extensive experiments on UCF101 human action and video summarization VSUMM datasets demonstrates the effectiveness of our proposed method.

show abstract

Section: Key Frame Detectionmentioning

confidence: 99%

Section: Key Frame Detectionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Self-Supervised Learning to Detect Key Frames in Videos

Gilani

Feng

Zhang

et al. 2020

Sensors

View full text Add to dashboard Cite

show abstract

“…Many earlier approaches relied on using a segmentation based pipline. Such methods typically extracted optical flow and SIFT features [35,18,9,15]. One of the first works [35,15] described a video with optical flow and detected local minimum changes in terms of similarity between successive frames.…”

Section: Related Workmentioning

confidence: 99%

Deep Keyframe Detection in Human Action Videos

Yan,

Gilani,

Qin

et al. 2018

Preprint

View full text Add to dashboard Cite

Detecting representative frames in videos based on human actions is quite challenging because of the combined factors of human pose in action and the background. This paper addresses this problem and formulates the key frame detection as one of finding the video frames that optimally maximally contribute to differentiating the underlying action category from all other categories. To this end, we introduce a deep two-stream ConvNet for key frame detection in videos that learns to directly predict the location of key frames. Our key idea is to automatically generate labeled data for the CNN learning using a supervised linear discriminant method. While the training data is generated taking many different human action videos into account, the trained CNN can predict the importance of frames from a single video. We specify a new ConvNet framework, consisting of a summarizer and discriminator. The summarizer is a two-stream ConvNet aimed at, first, capturing the appearance and motion features of video frames, and then encoding the obtained appearance and motion features for video representation. The discriminator is a fitting function aimed at distinguishing between the key frames and others in the video. We conduct experiments on a challenging human action dataset UCF101 and show that our method can detect key frames with high accuracy.

show abstract

“…e key objective of extracting key frames is to extract unique frames in a video and prepare the video sequences for quick processing [10]. In this paper, we propose an effective method for the extraction of a key frame from athlete sports video, which is accurate, fast, and efficient.…”

Section: Introductionmentioning

confidence: 99%

Key Frame Extraction for Sports Training Based on Improved Deep Learning

Tian

2021

Scientific Programming

View full text Add to dashboard Cite

With the rapid technological advances in sports, the number of athletics increases gradually. For sports professionals, it is obligatory to oversee and explore the athletics pose in athletes’ training. Key frame extraction of training videos plays a significant role to ease the analysis of sport training videos. This paper develops a sports actions’ classification system for accurately classifying athlete’s actions. The key video frames are extracted from the sports training video to highlight the distinct actions in sports training. Subsequently, a fully convolutional network (FCN) is used to extract the region of interest (ROI) pose detection of frames followed by the application of a convolution neural network (CNN) to estimate the pose probability of each frame. Moreover, a distinct key frame extraction approach is established to extract the key frames considering neighboring frames’ probability differences. The experimental results determine that the proposed method showed better performance and can recognize the athlete’s posture with an average classification rate of 98%. The experimental results and analysis validate that the proposed key frame extraction method outperforms its counterparts in key pose probability estimation and key pose extraction.

show abstract

Key frame extraction for salient activity recognition

Cited by 25 publications

References 21 publications

Self-Supervised Learning to Detect Key Frames in Videos

Self-Supervised Learning to Detect Key Frames in Videos

Deep Keyframe Detection in Human Action Videos

Key Frame Extraction for Sports Training Based on Improved Deep Learning

Contact Info

Product

Resources

About