Video stabilization using robust feature trajectories

Abstract

This paper proposes a new approach for video stabilization. Most existing video stabilization methods adopt a framework of three steps, motion estimation, motion compensation and image composition. Camera motion is often estimated based on pairwise registration between frames. Thus, these methods often assume static scenes or distant backgrounds. Furthermore, for scenes with moving objects, robust methods are required for finding the dominant motion. Such assumptions and judgements could lead to errors in motion parameters. Errors are compounded by motion compensation which smoothes motion parameters. This paper proposes a method to directly stabilize a video without explicitly estimating camera motion, thus assuming neither motion models nor dominant motion. The method first extracts robust feature trajectories from the input video. Optimization is then performed to find a set of transformations to smooth out these trajectories and stabilize the video. In addition, the optimization also considers quality of the stabilized video and selects a video with not only smooth camera motion but also less unfilled area after stabilization. Experiments show that our method can deal with complicated videos containing near, large and multiple moving objects.

Comparison of Motion Smoothing Strategies for Video Stabilization using Parametric Models

“…3D stabilization techniques are the most sophisticated ones as they reconstruct the camera trajectory through external camera calibration and an estimation of the 3D structure of the scene [7], [30], [52], [36], [15], or [61]. Several methods also combine 2D and 3D strategies [35], [15], for instance, by estimating only the epipolar geometry or involving extra information by depth cameras [38].…”

Digital video stabilization based on adaptive camera trajectory smoothing

“…Such mechanisms aim to remove highfrequency instability from camera motion [36]. Other approaches have focused on improving the quality of the videos, often lost in the stabilization process.…”

“…Approaches that use 2D transformations focus on contributing to specific steps in their stabilization process [36]. By considering the estimation of camera motion, 2D methods can be further subdivided into two categories [32]: (i) intensity-based approaches [37,38], which directly use the texture of the images as motion vector, and (ii) keypoint-based approaches [39,40], which locate a set of corresponding points in adjacent frames.…”

“…Since they usually have serious problems in handling large objects in the foreground [36], 2D methods are in general preferred in practice [36].…”

“…Other approaches have focused on improving the quality of the videos, often lost in the stabilization process. The most commonly used techniques include inpainting to fill missing frame parts [22,41,45], deconvolution to improve the video focus [22,41], and weighting of stabilization metrics and video quality aspects [36,45].…”