Interactive 3D video editing

Abstract

We present a generic and versatile framework for interactive editing of 3D video footage. Our framework combines the advantages of conventional 2D video editing with the power of more advanced, depth-enhanced 3D video streams. Our editor takes 3D video as input and writes both 2D or 3D video formats as output. Its underlying core data structure is a novel 4D spatio-temporal representation which we call the video hypervolume. Conceptually, the processing loop comprises three fundamental operators: slicing, selection, and editing. The slicing operator allows users to visualize arbitrary hyperslices from the 4D data set. The selection operator labels subsets of the footage for spatio-temporal editing. This operator includes a 4D graph-cut based algorithm for object selection.The actual editing operators include cut & paste, affine transformations, and compositing with other media, such as images and 2D video. For high-quality rendering, we employ EWA splatting with view-dependent texturing and boundary matting. We demonstrate the applicability of our methods to post-production of 3D video.

Interactive Video Segmentation via Local Appearance Model

“…A graph based on the four-dimensional representation is constructed between consecutive frames and then the graph cut algorithm is used to generate segmentation results. Similar approaches based on graph cut for video segmentation have been proposed with user interactions on single or multiple frames [9], [10], [23]- [26]. While these methods perform well in some sequences, they do not perform well in complex videos where objects undergo large shape deformation in cluttered backgrounds with diverse colors.…”

“…Waschbüsch et al [23] consider a video as a four-dimensional volume and use spatio-temporal coherence properties for segmentation. The object of interest is initialized with line strokes across several frames by users.…”

Future of 3DTV Broadcasting: The MUSCADE Perspective

“…Even though basic operations such as trimming and rearranging clips seem to be straightforward, a product aiming at mass audience has to be carefully authored due to the notorious discomfort issue associated with 3D video viewing [15]. Factors that aggravate discomfort such as excessive disparity and rapid depth changes have to be carefully considered while editing 3D footages [16].…”

Video SnapCut

“…Compared with global optimization techniques, Rotoscoping [Agarwala et al 2004;Stewart 2003] is a more localized technique for tracing partial foreground boundaries, however it cannot deal with rapid topology changes and often requires a significant amount of user interaction. In contrast to more standard keyframe-based user interfaces, treating video as a 3D volume and allowing the user to directly manipulate the spatio-temporal 3D video cube has gained considerable attention recently [Wang et al 2005;Waschbsch et al 2006]. However, in practice users find this UI unnatural and hard to use.…”