Video Summarization Based on Mutual Information and Entropy Sliding Window Method

Li, Wenlin; Qi, Deyu; Zhang, Changjian; Guo, Jing; Jiajun, Yao

doi:10.3390/e22111285

Cited by 8 publications

(4 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Inspired by the success of DGI, Zhu et al [20] introduced a new unsupervised graph contrastive learning framework, GRACE, which maximizes the consistency of node embeddings by masking node features and randomly deleting edges and using contrastive loss. Li et al [21] proposed the MIESW video summarization algorithm, which uses mutual information between frames as a measure, adjusts the size of the sliding window to group video content that is similar, and then extracts key frames based on frame entropy and average mutual information, thereby improving the robustness of the model and reducing the impact of noise.…”

Section: Graph Contrastive Learningmentioning

confidence: 99%

Video Summarization Generation Network Based on Dynamic Graph Contrastive Learning and Feature Fusion

Zhang,

Wu,

et al. 2024

Electronics

View full text Add to dashboard Cite

Video summarization aims to analyze the structure and content of videos and extract key segments to construct summarization that can accurately summarize the main content, allowing users to quickly access the core information without browsing the full video. However, existing methods have difficulties in capturing long-term dependencies when dealing with long videos. On the other hand, there is a large amount of noise in graph structures, which may lead to the influence of redundant information and is not conducive to the effective learning of video features. To solve the above problems, we propose a video summarization generation network based on dynamic graph contrastive learning and feature fusion, which mainly consists of three modules: feature extraction, video encoder, and feature fusion. Firstly, we compute the shot features and construct a dynamic graph by using the shot features as nodes of the graph and the similarity between the shot features as the weights of the edges. In the video encoder, we extract the temporal and structural features in the video using stacked L-G Blocks, where the L-G Block consists of a bidirectional long short-term memory network and a graph convolutional network. Then, the shallow-level features are obtained after processing by L-G Blocks. In order to remove the redundant information in the graph, graph contrastive learning is used to obtain the optimized deep-level features. Finally, to fully exploit the feature information of the video, a feature fusion gate using the gating mechanism is designed to fully fuse the shallow-level features with the deep-level features. Extensive experiments are conducted on two benchmark datasets, TVSum and SumMe, and the experimental results show that our proposed method outperforms most of the current state-of-the-art video summarization methods.

show abstract

Section: Graph Contrastive Learningmentioning

confidence: 99%

Video Summarization Generation Network Based on Dynamic Graph Contrastive Learning and Feature Fusion

Zhang,

Wu,

et al. 2024

Electronics

View full text Add to dashboard Cite

show abstract

“…7 The illustration of FLANN method with KD-Tree algorithm Illustration using the KD-Tree algorithm, for example, using a 2-dimensional descriptor. For example, there are 7 keypoints, with descriptor {(5,7), (3,4), (9,2), (1,2), (2,7), (6,1), (7,8)}. Descriptor (5,7) becomes the root, the next descriptor is placed in the left or right tree depending on the split part, as shown in Figure 7a.…”

Section: G Matching Feature Query Image With Keyframementioning

confidence: 99%

“…The recall value of the mutual information entropy method is the highest compared to other keyframe selection methods, 89.7%. Li et al [9] also use the mutual information entropy method for selecting a keyframe that produces a keyframe according to the video's main content.…”

Section: Introductionmentioning

confidence: 99%

Object Searching on Real-Time Video Using Oriented FAST and Rotated BRIEF Algorithm

Adhinata¹,

Harjoko²,

Wahyono³

2021

International Journal on Advanced Science, Engineering and Information Technology

View full text Add to dashboard Cite

The pre-processing and feature extraction stages are the primary stages in object searching on video data. Processing video in all frames is inefficient. Frames that have the same information should only be once processed to the next stage. Then, the feature extraction algorithm that is often used to process video frames is SIFT and SURF. The SIFT algorithm is very accurate but slow. On the other hand, the SURF algorithm is fast but less accurate. Therefore, the requirement for keyframe selection and feature extraction methods is fast and accurate in object searching on real-time video. Video is pre-processed by extracting video into frames. Then, the mutual information entropy method is used for keyframe selection. Keyframes are extracted using the ORB algorithm. The multiple object detection in the video is done by clustering on features. The feature extraction results on each cluster are matched with the results of the feature from the query image. Matching results from keyframe on video with the query image is used to retrieve the video's frame information. The experiment shows that keyframe selection is beneficial in real-time video data processing because the keyframe selection speed is faster than feature extraction on each frame. Then, feature extraction using the ORB algorithm results 2 times faster speed results than SIFT and SURF algorithms with values not so different from SIFT algorithm. This study's results can be developed as a security warning system in public places, especially by security in providing evidence of criminal cases from videos.

show abstract

“…However, this method still has high computational complexity in key frame selection. Lin et al 8 grouped the video sequence into frames with similar contents by using the mutual information between adjacent frames, and selected the frames whose mutual information is closed to the average as the video summary. Extracting keyframes by mutual information is fast and effective, so we extract keyframes by calculating the mutual information between adjacent frames.…”

Section: Introductionmentioning

confidence: 99%

No-reference stereoscopic video quality assessment based on Tchebichef moment

Chen

Wen²

2022

Fourteenth International Conference on Digital Image Processing (ICDIP 2022)

View full text Add to dashboard Cite

We propose a no-reference (NR) stereoscopic video quality assessment (SVQA) model based on Tchebichef moment in this paper. Specifically, we extract keyframes according to mutual information between adjacent frames, and then the extracted keyframes are segmented to patches to calculate low-order Tchebichef moments. Since the strong description ability of Tchebichef moment, and different order of Tchebichef moment can represent independent features with minimal information redundancy, we extract statistical features of Tchebichef moment on computed patches as spatial features. Considering the influence of distortions in spatiotemporal domain to video quality, we use the three-dimensional derivative of Gaussian filters to calculate the spatiotemporal energy responses and extract statistical features from the responses as spatiotemporal features. Finally, we combine the spatial and spatiotemporal features to predict the quality of stereoscopic videos. The proposed model is evaluated on the NAMA3DS1-COSPAD1, SVQA and Waterloo IVC phase I databases. The experimental results show that the proposed model achieved competitive performance as compared with existing SVQA models.

show abstract

Video Summarization Based on Mutual Information and Entropy Sliding Window Method

Cited by 8 publications

References 28 publications

Video Summarization Generation Network Based on Dynamic Graph Contrastive Learning and Feature Fusion

Video Summarization Generation Network Based on Dynamic Graph Contrastive Learning and Feature Fusion

Object Searching on Real-Time Video Using Oriented FAST and Rotated BRIEF Algorithm

No-reference stereoscopic video quality assessment based on Tchebichef moment

Contact Info

Product

Resources

About