Summarization of Egocentric Videos: A Comprehensive Survey

Molino, Ana García del; Tan, Cheston; Lim, Joo‐Hwee; Tan, Ah-Hwee

doi:10.1109/thms.2016.2623480

Cited by 70 publications

(52 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A hierarchical model has been proposed to learn with few labels, and it is optimized to generate video summary containing interesting objects [30]. Egocentric videos [31] can be compacted with importance of people and objects [8]; on the other hand, Zheng et al explicitly consider how one sub-event leads to another in order to provide a better sense of story for those kinds of videos [9]. Meanwhile, Yao et al propose a pairwise deep ranking model to highlight video segments of first-person videos [32].…”

Section: Video Summarizationmentioning

confidence: 99%

How Local Is the Local Diversity? Reinforcing Sequential Determinantal Point Processes with Dynamic Ground Sets for Supervised Video Summarization

Wang

Yang

et al. 2018

Computer Vision – ECCV 2018

View full text Add to dashboard Cite

The large volume of video content and high viewing frequency demand automatic video summarization algorithms, of which a key property is the capability of modeling diversity. If videos are lengthy like hours-long egocentric videos, it is necessary to track the temporal structures of the videos and enforce local diversity. The local diversity refers to that the shots selected from a short time duration are diverse but visually similar shots are allowed to co-exist in the summary if they appear far apart in the video. In this paper, we propose a novel probabilistic model, built upon SeqDPP [1], to dynamically control the time span of a video segment upon which the local diversity is imposed. In particular, we enable SeqDPP to learn to automatically infer how local the local diversity is supposed to be from the input video. The resulting model is extremely involved to train by the hallmark maximum likelihood estimation (MLE), which further suffers from the exposure bias and non-differentiable evaluation metrics. To tackle these problems, we instead devise a reinforcement learning algorithm for training the proposed model. Extensive experiments verify the advantages of our model and the new learning algorithm over MLE-based methods.

show abstract

Section: Video Summarizationmentioning

confidence: 99%

How Local Is the Local Diversity? Reinforcing Sequential Determinantal Point Processes with Dynamic Ground Sets for Supervised Video Summarization

Wang

Yang

et al. 2018

Computer Vision – ECCV 2018

View full text Add to dashboard Cite

show abstract

“…Long-running and boring videos decrease the propensity of future viewers to watch the footage, even the recorders could not pay attention to the majority of recordings [2], making significant moments to be lost along with activities that do not merit recording. Thus, a central challenge is to This work relates to an Ph.D. thesis.…”

Section: Introductionmentioning

confidence: 99%

Semantic Hyperlapse: a Sparse Coding-based and Multi-Importance Approach for First-Person Videos

Silva

Campos

2019

Anais Estendidos Do XXXII Conference on Graphics, Patterns and Images (SIBRAPI Estendido 2019)

View full text Add to dashboard Cite

The availability of low-cost, high-quality personal wearable cameras combined with the unlimited storage capacity of video-sharing websites has evoked a growing interest in First-Person Videos (FPVs). Such videos are usually composed of long-running unedited streams captured by a device attached to the user body, which makes them tedious and visually unpleasant to watch. Consequently, there is a rise in the need to provide quick access to the information therein. To address this need, efforts have been applied to the development of techniques such as Hyperlapse and Semantic Hyperlapse, which aims to create visually pleasant shorter videos and emphasize semantic portions of the video, respectively. The state-of-the-art Semantic Hyperlapse method SSFF, negligees the level of importance of the relevant information, by only evaluating if it is significant or not. Other limitations of SSFF are the number of input parameters, the scalability in the number of visual features to describe the frames, and the abrupt change in the speed-up rate of consecutive video segments. In this dissertation, we propose a parameter-free Sparse Coding based methodology to adaptively fast-forward First-Person Videos, that emphasize the semantic portions applying a multi-importance approach. Experimental evaluations show that the proposed method creates shorter version video retaining more semantic information, with fewer abrupt transitions of speed-up rates, and more stable final videos than the output of SSFF. Visual results and graphical explanation of the methodology can be visualized through the link: https://youtu.be/8uStih8P5-Y.

show abstract

“…A central challenge is to selective highlight the meaningful parts of the videos without losing the whole message that the video should convey. Although video summarization techniques [4,14] provide quick access to videos' information, they only return segmented clips or single images of the relevant moments. By not including the very last and the following frames of a clip, the summarization might lose the clip context [20].…”

Section: Introductionmentioning

confidence: 99%

A Weighted Sparse Sampling and Smoothing Frame Transition Approach for Semantic Fast-Forward First-Person Videos

Silva

Ramos

Ferreira

et al. 2018

2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition

View full text Add to dashboard Cite

Thanks to the advances in the technology of low-cost digital cameras and the popularity of the self-recording culture, the amount of visual data on the Internet is going to the opposite side of the available time and patience of the users. Thus, most of the uploaded videos are doomed to be forgotten and unwatched in a computer folder or website. In this work, we address the problem of creating smooth fast-forward videos without losing the relevant content. We present a new adaptive frame selection formulated as a weighted minimum reconstruction problem, which combined with a smoothing frame transition method accelerates first-person videos emphasizing the relevant segments and avoids visual discontinuities. The experiments show that our method is able to fast-forward videos to retain as much relevant information and smoothness as the state-ofthe-art techniques in less time. We also present a new 80hour multimodal (RGB-D, IMU, and GPS) dataset of firstperson videos with annotations for recorder profile, frame scene, activities, interaction, and attention 1 .

show abstract

Summarization of Egocentric Videos: A Comprehensive Survey

Cited by 70 publications

References 42 publications

How Local Is the Local Diversity? Reinforcing Sequential Determinantal Point Processes with Dynamic Ground Sets for Supervised Video Summarization

How Local Is the Local Diversity? Reinforcing Sequential Determinantal Point Processes with Dynamic Ground Sets for Supervised Video Summarization

Semantic Hyperlapse: a Sparse Coding-based and Multi-Importance Approach for First-Person Videos

A Weighted Sparse Sampling and Smoothing Frame Transition Approach for Semantic Fast-Forward First-Person Videos

Contact Info

Product

Resources

About