Slippage-free background replacement for hand-held video

Abstract

Figure 1: Given a video with a pre-segmented foreground subject, our method can replace the original background with a background from another video, even when both videos are captured using a hand-held camera with different motion. One of our main goals is to avoid the AbstractWe introduce a method for replacing the background in a video of a moving foreground subject, when both the source video capturing the subject, and the target video capturing the new background scene, are natural videos, casually captured using a freely moving hand-held camera. We assume that the foreground subject has already been extracted, and focus on the challenging task of generating a video with a new background, such that the new background motion appears compatible with the original one. Failure to match the motion results in disturbing slippage or moonwalk artifacts, where the subject's feet appear to slide or slip over the ground. While matching the motion across the entire frame is impossible for scenes with differing geometry, we aim to match the local motion of the ground in the vicinity of the subject. This is achieved by reordering and warping the available target background frames in a manner that optimizes a suitably designed objective function.

Interactive Videos: Plausible Video Editing using Sparse Structure Points

“…The wide availability of portable and affordable video recorders enables the unrestricted access to the video contents in daily life. This has motivated a large body of research to develop advanced video editing tools in video compositing [SE02, XCF06, GGC*08, ZDJ*09, RWSG13, ZYQ*14] and producing after effects, such as bullet‐time [ZDJ*09], action shots [KWB*15], etc. These works share a common goal of achieving plausible video editing without explicitly reconstructing the 3D geometries of the scenes.…”

“…Zhong et al . [ZYQ*14] introduce a system that uses only estimated 2D transformations between consecutive frames to replace the background in a video of a moving foreground subject without suffering from slippage artifacts. Zhang et al .…”

Towards high-quality 3D telepresence with commodity RGBD camera

“…Date et al [15] introduced a highly realistic 3D display system that displays a person in a remote location as a life-size stereoscopic image against background scenery, which corresponds to the observer's viewing position to well reproduce the fidelity of existence. Zhong et al [99] focused on the challenging task of generating a video with a new background, such that the new background motion appears compatible with the original one. One common limitation of above works is their lack of attention on the natural and consistent composition of the scene space (geometry-wise).…”