3D video recorder

Abstract

We present the 3D Video Recorder, a system capable of recording, processing, and playing three-dimensional video from multiple points of view. We first record 2D video streams from several synchronized digital video cameras and store pre-processed images to disk. An off-line processing stage converts these images into a time-varying threedimensional hierarchical point-based data structure and stores this 3D video to disk. We show how we can trade-off 3D video quality with processing performance and devise efficient compression and coding schemes for our novel 3D video representation. A typical sequence is encoded at less than 7 megabit per second at a frame rate of 8.5 frames per second. The 3D video player decodes and renders 3D videos from hard-disk in real-time, providing interaction features known from common video cassette recorders, like variable-speed forward and reverse, and slow motion. 3D video playback can be enhanced with novel 3D video effects such as freeze-and-rotate and arbitrary scaling. The player builds upon point-based rendering techniques and is thus capable of rendering high-quality images in real-time. Finally, we demonstrate the 3D Video Recorder on multiple real-life video sequences.

Point Cloud Streaming for 3D Avatar Communication

“…Some papers (Wu & Matsuyama, 2003), (Ueda et al, 2004), (Würmlin et al, 2002), (Würmlin et al, 2004), have already presented systems reconstructing 3D images in real-time. In many of these methods, first object silhouettes are extracted from multiple images obtained from several viewpoints.…”

Video-based Capturing and Rendering of People

“…Shape-from-silhouette methods reconstruct geometry models of a scene from multi-view silhouette images or video streams. Examples are image-based [27,51] or polyhedral visual hull methods [26], as well as approaches performing point-based reconstruction [16]. The combination of stereo reconstruction with visual hull rendering leads to a more faithful reconstruction of surface concavities [25].…”

Combining 3D flow fields with silhouette-based human motion capture for immersive video

“…In 3D video, dynamic models of scenes that were recorded from several camera perspectives are reconstructed and rendered from novel viewpoints. Examples of methods that were used to approach that problem are shape-from silhouette-like approaches, such as visual hull [21,37], stereo-based approaches [23] or a combination of both [27]. Ray-space methods from image-based rendering that approximate the plenoptic function [1] in a scene, such as the lightfield [18] or the lumigraph [11], can also be considered to be part of the same effort.…”