Hardware‐Accelerated Rendering of Photo Hulls

Abstract

This paper presents an efficient hardware‐accelerated method for novel view synthesis from a set of images or videos. Our method is based on the photo hull representation, which is the maximal photo‐consistent shape. We avoid the explicit reconstruction of photo hulls by adopting a view‐dependent plane‐sweeping strategy. From the target viewpoint slicing planes are rendered with reference views projected onto them. Graphics hardware is exploited to verify the photo‐consistency of each rasterized fragment. Visibilities with respect to reference views are properly modeled, and only photo‐consistent fragments are kept and colored in the target view. We present experiments with real images and animation sequences. Thanks to the more accurate shape of the photo hull representation, our method generates more realistic rendering results than methods based on visual hulls. Currently, we achieve rendering frame rates of 2–3 fps. Compared to a pure software implementation, the performance of our hardware‐accelerated method is approximately 7 times faster. Categories and Subject Descriptors (according to ACM CCS): CR Categories: I.3.3 [Computer Graphics]: Picture/Image Generation; I.3.7 [Computer Graphics]: Three‐Dimensional Graphics and Realism.

A Flexible Client-Driven 3DTV System for Real-Time Acquisition, Transmission, and Display of Dynamic Scenes

“…Some other hardware-aided method can give good results [26], effectively combing a plane-sweeping algorithm with view synthesis for real-time 3D scene acquisition. Reference [27] explicitly reconstructs photo hulls by adopting a viewdependent plane-sweeping strategy. Graphics hardware is exploited to verify the photo-consistency of each rasterized fragment.…”

Interactive Collision Detection for Free-Viewpoint Video

“…Since our technique is fairly simple, and works directly on the video frame data, we are free to choose any existing free viewpoint video reconstruction and rendering technique, such as the voxel-based visual hull calculation by Hasenfratz et al (Hasenfratz et al, 2003) or the real-time polygonal visual hull calculation by Matusik et al (Matusik et al, 2001). For our experiments, we implemented the photohull calculation technique by Li et al (Li et al, 2004).…”

“…The images along with the silhouette images are gathered on a central PC which performs the rigid body simulation, the collision detection and the visualization. The real-world scene is rendered by the Photohull plane sweeping algorithm by Li et al (Li et al, 2004) and runs entirely on the GPU. We coupled our collision detection system with a rigid body simulator (Open Dynamics Engine (ODE, 2006)).…”

Modulating the Size of Backprojection Surface Patches, in Volumetric Stereo, for Increasing Reconstruction Accuracy and Robustness

“…The first regards space sweeping algorithms, where a single hypothetical surface is swept along depth [12,13,14,15,16,17]. The orientation of the sweeping surface is a priori set independently to the actual structure of the imaged scene.…”