Full-frame video stabilization with motion inpainting

Matsushita, Yasuyuki; Ofek, Eyal; Ge, Weina; Tang, Xiaoou; Shum, Heung-Yeung

doi:10.1109/tpami.2006.141

Cited by 493 publications

(350 citation statements)

References 31 publications

Supporting

Mentioning

344

Contrasting

Unclassified

Order By: Relevance

“…In [Gleicher and Liu 2008] the stabilized camera motion is computed by interpolating between homography transformations using matrix logarithms. Matsushita et al [2006] discuss additional important aspects of video stabilization, such as extending the stabilized frames to become full-frames and reducing the motion blur.…”

Section: Introductionmentioning

confidence: 99%

Video stabilization using epipolar geometry

2012

View full text Add to dashboard Cite

We present a new video stabilization technique that uses projective scene reconstruction to treat jittered video sequences. Unlike methods that recover the full three-dimensional geometry of the scene, this model accounts for simple geometric relations between points and epipolar lines. Using this level of scene understanding, we obtain the physical correctness of 3D stabilization methods yet avoid their lack of robustness and computational costs. Our method consists of tracking feature points in the scene and using them to compute fundamental matrices that model stabilized camera motion. We then project the tracked points onto the novel stabilized frames using epipolar point transfer and synthesize new frames using image-based frame warping. Since this model is only valid for static scenes, we develop a time-view reprojection that accounts for non-stationary points in a principled way. This reprojection is based on modeling the dynamics of smooth inertial object motion in three-dimensional space and allows us to avoid the need to interpolate stabilization for moving objects from their static surrounding. Thus, we achieve an adequate stabilization when both the camera and the objects are moving. We demonstrate the abilities of our approach to stabilize hand-held video shots in various scenarios: scenes with no parallax that challenge 3D approaches, scenes containing non-trivial parallax effects, videos with camera zooming and in-camera stabilization, as well as movies with large moving objects.

show abstract

Section: Introductionmentioning

confidence: 99%

Video stabilization using epipolar geometry

2012

View full text Add to dashboard Cite

show abstract

“…Xiang et al [22] showed how to utilize the video retargeting method to improve the composition of video contents. Temporal information [23] and camera stability [24] were also considered to improve video aesthetic quality. More recently, Lu et al proposed a spatio-temporally consistent color and structure optimization method [25].…”

Section: Related Workmentioning

confidence: 99%

Detecting and Removing Visual Distractors for Video Aesthetic Enhancement

Zhang

et al. 2018

IEEE Trans. Multimedia

View full text Add to dashboard Cite

Abstract-Personal videos often contain visual distractors, which are objects that are accidentally captured that can distract viewers from focusing on the main subjects. We propose a method to automatically detect and localize these distractors through learning from a manually labeled dataset. To achieve spatially and temporally coherent detection, we propose extracting features at the Temporal-Superpixel (TSP) level using a traditional SVM-based learning framework. We also experiment with end-to-end learning using Convolutional Neural Networks (CNNs), which achieves slightly higher performance than other methods. The classification result is further refined in a post-processing step based on graph-cut optimization. Experimental results show that our method achieves an accuracy of 81% and a recall of 86%. We demonstrate several ways of removing the detected distractors to improve the video quality, including video hole filling; video frame replacement; and camera path re-planning. The user study results show that our method can significantly improve the aesthetic quality of videos.

show abstract

“…Motion denoising has been addressed before in a global manner, known as video stabilization (e.g. [11,10]). Camera jitter (mostly from hand-held devices) is modeled using image-level transforms between consecutive frames, which are used to synthesize a sequence with smoother camera motion.…”

Section: Background and Related Workmentioning

confidence: 99%