Static video summarization using multi-CNN with sparse autoencoder and random forest classifier

Nair, Madhu S.; Mohan, Jesna

doi:10.1007/s11760-020-01791-4

Cited by 21 publications

(18 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The same backbone GoogleNet and ResNet-50 are used in [74] for multi-stage networks for video summarization. In the same context, and using Sparse Autoencoders network with Random Forest Classifier, the authors in [75] proposed a CNN-based model for key-frames selection. A set of backbones, including AlexNet, GoogleNet, VGG-16, and Inception-ResNet-v2, have been used then compared the impact of each one on the summarization results using VSUMM and OVP datasets.…”

Section: Video Summarizationmentioning

confidence: 99%

Backbones-Review: Feature Extraction Networks for Deep Learning and Deep Reinforcement Learning Approaches

Elharrouss¹,

Akbari²,

Almaadeed³

et al. 2022

Preprint

View full text Add to dashboard Cite

To understand the real world using various types of data, Artificial Intelligence (AI) is the most used technique nowadays. While finding the pattern within the analyzed data represents the main task. This is performed by extracting representative features step, which is proceeded using the statistical algorithms or using some specific filters. However, the selection of useful features from large-scale data represented a crucial challenge. Now, with the development of convolution neural networks (CNNs), the feature extraction operation has become more automatic and easier. CNNs allow to work on large-scale size of data, as well as cover different scenarios for a specific task. For computer vision tasks, convolutional networks are used to extract features also for the other parts of a deep learning model. The selection of a suitable network for feature extraction or the other parts of a DL model is not random work. So, the implementation of such a model can be related to the target task as well as the computational complexity of it. Many networks have been proposed and become the famous networks used for any DL models in any AI task. These networks are exploited for feature extraction or at the beginning of any DL model which is named backbones. A backbone is a known network trained in many other tasks before and demonstrates its effectiveness. In this paper, an overview of the existing backbones, e.g. VGGs, ResNets, DenseNet, etc, is given with a detailed description. Also, a couple of computer vision tasks are discussed by providing a review of each task regarding the backbones used. In addition, a comparison in terms of performance is also provided, based on the backbone used for each task.

show abstract

Section: Video Summarizationmentioning

confidence: 99%

Backbones-Review: Feature Extraction Networks for Deep Learning and Deep Reinforcement Learning Approaches

Elharrouss¹,

Akbari²,

Almaadeed³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Video summarization was performed by [21] on the Internet of Things (IoT) surveillance domain by designing a CNN framework that performs shot segmentation and image memorability, and aesthetic and entropy features are used to maintain the diversity of the summary. The authors in [22] used a sparse autoencoder that combines feature vectors derived from four famous image CNNs into a reduced space and a random forest classifier to select key frames.…”

Section: B Cnn-basedmentioning

confidence: 99%

“…In [22] it was mentioned that redundant frames increase the complexity of detecting keyframes. A redundant frame is defined as one that is identical or very similar to the previous frame.…”

Section: Elimination Of Similar Framesmentioning

confidence: 99%

“…In [22] this approach was used for elimination, which is based on the work reported in [39]. Briefly, each frame in a video is converted to a SIFT image, where each pixel is represented by a 128-D SIFT descriptor.…”

Section: Elimination Of Similar Framesmentioning

confidence: 99%

“…As mentioned previously, in this study, we propose the use of HEVC features for video summarization, and we also use CNN features generated from the obtained networks, as proposed by [22]. This will allow us to compare our work with existing solutions and it will allow us to test the suitability of combining the proposed HEVC features with CNN features.…”

Section: Dimensionality Reductionmentioning

confidence: 99%

See 2 more Smart Citations

CNN and HEVC Video Coding Features for Static Video Summarization

Issa

Shanableh

2022

IEEE Access

View full text Add to dashboard Cite

This study proposes a novel solution for the detection of keyframes for static video summarization. We preprocessed the well-known video datasets by coding them using the HEVC video coding standard. During coding, 64 proposed features were generated from the coder for each frame. Additionally, we converted the original YUVs of the raw videos into RGB images and fed them into pretrained CNN networks for feature extraction. These include GoogleNet, AlexNet, Inception-ResNet-v2, and VGG16. The modified datasets are made publicly available to the research community. Before detecting keyframes in a video, it is important to identify and eliminate duplicate or similar video frames. A subset of the proposed HEVC feature set was used to identify these frames and eliminate them from the video. We also propose an elimination solution based on the sum of the absolute differences between a frame and its motion-compensated predecessor. The proposed solutions are compared with existing works based on an SIFT flow algorithm that uses CNN features. Subsequently, an optional dimensionality reduction based on stepwise regression was applied to the feature vectors prior to detecting key frames. The proposed solution is compared with existing studies that use sparse autoencoders with CNN features for dimensionality reduction. The accuracy of the proposed key-frame detection system was assessed using the positive predictive values, sensitivity, and F-scores. Combining the proposed solution with Multi-CNN features and using a random forest classifier, it was shown that the proposed solution achieved an average F-score of 0.98.

show abstract

Social-Media Video Summarization Using Convolutional Neural Network and Kohnen’s Self Organizing Map

Kashid,

Awasthi,

Kumar

et al. 2024

Lecture Notes in Networks and Systems

View full text Add to dashboard Cite

Static video summarization using multi-CNN with sparse autoencoder and random forest classifier

Cited by 21 publications

References 30 publications

Backbones-Review: Feature Extraction Networks for Deep Learning and Deep Reinforcement Learning Approaches

Backbones-Review: Feature Extraction Networks for Deep Learning and Deep Reinforcement Learning Approaches

CNN and HEVC Video Coding Features for Static Video Summarization

Social-Media Video Summarization Using Convolutional Neural Network and Kohnen’s Self Organizing Map

Contact Info

Product

Resources

About