Novel‐View Synthesis of Outdoor Sport Events Using an Adaptive View‐Dependent Geometry

Abstract

Figure 1: Novel view rendering of a soccer scene using only the two input cameras shown to the left and the right. AbstractWe propose a novel fully automatic method for novel-viewpoint synthesis. Our method robustly handles multicamera setups featuring wide-baselines in an uncontrolled environment. In a first step, robust and sparse point correspondences are found based on an extension of the Daisy features [TLF10]. These correspondences together with back-projection errors are used to drive a novel adaptive coarse to fine reconstruction method, allowing to approximate detailed geometry while avoiding an extreme triangle count. To render the scene from arbitrary viewpoints we use a view-dependent blending of color information in combination with a view-dependent geometry morph. The view-dependent geometry compensates for misalignments caused by calibration errors. We demonstrate that our method works well under arbitrary lighting conditions with as little as two cameras featuring wide-baselines. The footage taken from real sports broadcast events contains fine geometric structures, which result in nice novel-viewpoint renderings despite of the low resolution in the images.

Slippage-free background replacement for hand-held video

“…For this example we find that our method can tolerate an angular difference up to 15 degrees, while larger differences introduce undesirable rotation and blank regions. Note that although video-based novel view synthesis has been extensively studied [Germann et al 2012], this remains a difficult problem in general scenes.…”

“…Video-Based Rendering methods typically represent a dynamic scene as a collection of video streams with correspondences among them, allowing to re-render the performance from novel views (e.g., [Stich et al 2008;Germann et al 2012]). In our case, the source and the target videos are captured in different scenes, and our goal is to replay the performance from the same (moving) point of view, but using a different background.…”

The Stereoscopic Zoom

“…The main idea of freeviewpoint video is that multiple images of a scene can be projected onto a geometric proxy in order to generate new realistic view-dependent images [18,26,23,5,41]. Most contributions in this domain target the productions of live events [13]. Stereoscopic images can be rendered from a standard camera configuration used for a 2D broadcast [15], i.e.…”

Correspondence and Depth-Image Based Rendering a Hybrid Approach for Free-Viewpoint Video

“…Billboard approximations of moving actors have been used successfully for navigating through multiview data of moving actors [2]. For free-viewpoint video football replays, a billboard cloud has been used to approximate the individual players [15], [16]. For such data sets, it is often infeasible to reconstruct an accurate per-pixel depth.…”