EgoSampling: Fast-forward and stereo for egocentric videos

Poleg, Yair; Halperin, Tavi; Arora, Chetan; Peleg, Bezalel

doi:10.1109/cvpr.2015.7299109

Cited by 64 publications

(94 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In concurrent work, Poleg et al [2014], as in this work, carefully sample frames on a video, particularly in semi-regular oscillating videos such as those captured when walking, to select frames leading to more stable timelapse results. They also suggest producing two simultaneous, but offset, tracks to automatically extract stereo pairs of video.…”

Section: Timelapse and Hyperlapse Methodsmentioning

confidence: 98%

Real-time hyperlapse creation via optimal frame selection

et al. 2015

View full text Add to dashboard Cite

Naive hyperlapse Our approach MeanStandard Deviation Naive Ours Figure 1: Hand-held videos often exhibit significant semi-regular high-frequency camera motion due to, for example, running (dotted blue line). This example shows how a naive 8x hyperlapse (i.e., keeping 1 out every 8 frames) results in frames with little overlap that are hard to align (black lines). By allowing small violations of the target skip rate we create hyperlapse videos that are smooth even when there is significant camera motion (pink lines). Optimizing an energy function (color-coded in Middle image) that balances matching the target rate while minimizing frame-to-frame motion results in a set of frames that are then stabilized. (Right) To illustrate the alignment we show the mean and standard deviation of three successive frames (in red box on the Left plot) after stabilization for the naive hyperlapse (Top Right) and our result (Bottom Right) -these show that our selected frames align much better than those from naive selection. AbstractLong videos can be played much faster than real-time by recording only one frame per second or by dropping all but one frame each second, i.e., by creating a timelapse. Unstable hand-held moving videos can be stabilized with a number of recently described methods. Unfortunately, creating a stabilized timelapse, or hyperlapse, cannot be achieved through a simple combination of these two methods. Two hyperlapse methods have been previously demonstrated: one with high computational complexity and one requiring special sensors. We present an algorithm for creating hyperlapse videos that can handle significant high-frequency camera motion and runs in real-time on HD video. Our approach does not require sensor data, thus can be run on videos captured on any camera. We optimally select frames from the input video that best match a desired target speed-up while also resulting in the smoothest possible camera motion. We evaluate our approach using several input videos from a range of cameras and compare these results to existing methods.

show abstract

Section: Timelapse and Hyperlapse Methodsmentioning

confidence: 98%

Real-time hyperlapse creation via optimal frame selection

et al. 2015

View full text Add to dashboard Cite

show abstract

“…A popular way to compress videos into shorter clips is to fast-forward them, and timelapse and hyperlapse are two appealing techniques to accomplish this; the former handles videos captured using static (or slowmoving) cameras over a long period of time (e.g.a day-to-night landscape shown in one minute), while the latter is applied to videos captured by moving (often hand-held) cameras that covers large distances (e.g.a hike across the Great Wall summarized in one minute). These videos are created by sampling only a subset of the frames (either uniformly or taking video features into account [JKT * 15, PHAP15]).…”

Section: Introductionmentioning

confidence: 99%

Photometric Stabilization for Fast‐forward Videos

Zhang

Lee

Sunkavalli

et al. 2017

Computer Graphics Forum

View full text Add to dashboard Cite

Videos captured by consumer cameras often exhibit temporal variations in color and tone that are caused by camera autoadjustments like white-balance and exposure. When such videos are sub-sampled to play fast-forward, as in the increasingly popular forms of timelapse and hyperlapse videos, these temporal variations are exacerbated and appear as visually disturbing high frequency flickering. Previous techniques to photometrically stabilize videos typically rely on computing dense correspondences between video frames, and use these correspondences to remove all color changes in the video sequences. However, this approach is limited in fast-forward videos that often have large content changes and also might exhibit changes in scene illumination that should be preserved. In this work, we propose a novel photometric stabilization algorithm for fast-forward videos that is robust to large content-variation across frames. We compute pairwise color and tone transformations between neighboring frames and smooth these pair-wise transformations while taking in account the possibility of scene/content variations. This allows us to eliminate high-frequency fluctuations, while still adapting to real variations in scene characteristics. We evaluate our technique on a new dataset consisting of controlled synthetic and real videos, and demonstrate that our techniques outperforms the state-of-the-art.

show abstract

“…More recent methods have adopted the optimization of frame selection [8], [16], [5]. Poleg et al [16] focus on an adaptive frame selection based on minimizing an energy function.…”

Section: Related Workmentioning

confidence: 99%

“…Poleg et al [16] focus on an adaptive frame selection based on minimizing an energy function. They modeled the video as a graph by mapping the frames as the nodes and the edges weight reflecting the cost of the transition between the frames in the final video.…”

Section: Related Workmentioning

confidence: 99%

Towards Semantic Fast-Forward and Stabilized Egocentric Videos

Silva

Ramos

Ferreira

et al. 2016

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Abstract. The emergence of low-cost personal mobiles devices and wearable cameras and the increasing storage capacity of video-sharing websites have pushed forward a growing interest towards first-person videos. Since most of the recorded videos compose long-running streams with unedited content, they are tedious and unpleasant to watch. The fastforward state-of-the-art methods are facing challenges of balancing the smoothness of the video and the emphasis in the relevant frames given a speed-up rate. In this work, we present a methodology capable of summarizing and stabilizing egocentric videos by extracting the semantic information from the frames. This paper also describes a dataset collection with several semantically labeled videos and introduces a new smoothness evaluation metric for egocentric videos that is used to test our method.

show abstract

EgoSampling: Fast-forward and stereo for egocentric videos

Cited by 64 publications

References 21 publications

Real-time hyperlapse creation via optimal frame selection

Real-time hyperlapse creation via optimal frame selection

Photometric Stabilization for Fast‐forward Videos

Towards Semantic Fast-Forward and Stabilized Egocentric Videos

Contact Info

Product

Resources

About