A bottom-up summarization algorithm for videos in the wild

Pan, Gang; Zheng, Yaoxian; Zhang, Rufei; Han, Zhenjun; Sun, Di; Qu, Xingming

doi:10.1186/s13634-019-0611-y

Cited by 13 publications

(8 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These works require the use of fixed cameras and suffer from immobility, significantly restricting the range of potential applications. The requirement of using obtrusive and non-wearable sensors to extract a user's emotions has encouraged most research to focus on direct affective content utilizing video frames [12][13][14][15], video segments [16][17][18][19], audio and text features [14]. The present results remained poor on average due to the semantic gap problem [10] of detected objects or events.…”

Section: Introductionmentioning

confidence: 75%

“…In contrast to previous works, where video highlight/summaries are generated with a single score function representing frame-level importance [2,10,[12][13][14][15]18,19], we trained two independent classifiers on highlight start and end times. This divide-and-conquer approach simplifies the task for each classifier by specializing in only one classification ( Figure 12).…”

Section: Machine Learning Approachmentioning

confidence: 99%

See 1 more Smart Citation

Intelligent Video Highlights Generation with Front-Camera Emotion Sensing

Meyer

Wei

Jiang

2021

Sensors

View full text Add to dashboard Cite

In this paper, we present HOMER, a cloud-based system for video highlight generation which enables the automated, relevant, and flexible segmentation of videos. Our system outperforms state-of-the-art solutions by fusing internal video content-based features with the user’s emotion data. While current research mainly focuses on creating video summaries without the use of affective data, our solution achieves the subjective task of detecting highlights by leveraging human emotions. In two separate experiments, including videos filmed with a dual camera setup, and home videos randomly picked from Microsoft’s Video Titles in the Wild (VTW) dataset, HOMER demonstrates an improvement of up to 38% in F1-score from baseline, while not requiring any external hardware. We demonstrated both the portability and scalability of HOMER through the implementation of two smartphone applications.

show abstract

Section: Introductionmentioning

confidence: 75%

Section: Machine Learning Approachmentioning

confidence: 99%

Intelligent Video Highlights Generation with Front-Camera Emotion Sensing

Meyer

Wei

Jiang

2021

Sensors

View full text Add to dashboard Cite

show abstract

“…when comparing various models, CNN using only TDM characteristics is the superior option. It is claimed in Pan et al [2] that energy entropy-based summarization can be used to further refine this complicated model. Assigning an entropy block to each paired frame, we may calculate their energy using the following formulas:…”

Section: Literature Reviewmentioning

confidence: 99%

Statistical Evaluation of Video Summarization Models from an Empirical Perspective

Kaur¹,

Mishra²

2023

View full text Add to dashboard Cite

Video summarization is the process of creating a shortened version of a longer video while retaining its essential content and meaning. It entails automatically identifying the most important parts of the video and selecting the relevant frames, shots, or scenes that best represent the original video's content. Video summarization entails complex image signal analysis and processing to extract the most important frames or shots from a video while discarding redundant or less informative ones. Several stages of analysis and processing are typically involved in the process, which may include video segmentation, feature extraction, frame selection, classification, and quality assessment. A variety of algorithms and system models are available for this task. Classification architectures such as convolutional neural networks, recurrent neural networks, and others are used to categorize video frames as redundant or non-redundant. This article provides a categorization and analysis of video summarization methodologies, with a focus on methods from the real-time video summarizing (RVS) domain. The current study will aid in laying the groundwork for future research and investigating potential research avenues by combining key research findings and data for quick reference. Video summarization has been shown to be useful in a variety of real-world contexts in smart cities, such as detecting anomalies in a video surveillance system. To address this issue, research studies can be conducted to evaluate and compare different video summarization algorithms in terms of their effectiveness, efficiency, and suitability for various applications. These studies can use benchmark datasets and standardized evaluation metrics to provide objective and quantitative comparisons of different algorithms. Based on the findings of these studies, researchers and multimedia system designers can make informed decisions about which algorithmic combination will work best for their application.

show abstract

“…Video outline techniques ordinarily recognize frames that have higher development esteems, and thusly moving article recognition can be a component vector for this reason. The work in [1] utilizes Moving Object Detection and Image Similarity with the end goal of KFE. It utilizes ViBe calculation and breakers between frame distinction strategy by isolating the first video into a few portions that contain the moving item.…”

Section: Previous Work On Video Summarization Modelsmentioning

confidence: 99%

Estimation of concise video summaries from long sequence videos using deep learning via LSTM

Kaur

Mishra

2022

ijhs

View full text Add to dashboard Cite

Video summarization is a field of signal processing which deals with removing redundant information from a sequence of frames. In order to design an efficient video summarization model, it is imperative that frame features must be analyzed, and their similarity must be evaluated. This process would assist in aggregation of frames that have similar feature sets, and then use this information to reduce frame redundancy. Design of an efficient feature extraction and analysis model must be done in order to improve efficiency of video summarization. Models that aim at improving this efficiency, tend to improve the computational complexity of summarization, thereby limiting their applicability. In order to remove this drawback, The proposed event-based model for long video summarization uses a combination of LSTM based CNN with feature variance method for estimation of keyframes. The model works is 2-phases, wherein in the first phase various events are recognized from the video using LSTM based CNN model; while in the second phase these events are individually summarized using variance-based threshold engine. The proposed event classification-based LSTM CNN model with variance-based summarization outperforms other models, and achieves 8% improvement in terms of compression ratio.

show abstract

A bottom-up summarization algorithm for videos in the wild

Cited by 13 publications

References 31 publications

Intelligent Video Highlights Generation with Front-Camera Emotion Sensing

Intelligent Video Highlights Generation with Front-Camera Emotion Sensing

Statistical Evaluation of Video Summarization Models from an Empirical Perspective

Estimation of concise video summaries from long sequence videos using deep learning via LSTM

Contact Info

Product

Resources

About