Zhou et al. Ours 1× Input Cameras 30× Extrapolated Right Extrapolated Left Zhou Figure 1: We propose a novel view synthesis method that can generate extreme views, i.e., images synthesized from a small number of cameras (two in this example) and from significantly different viewpoints. In this comparison with the method by Zhou et al.[34], we show the left view from the camera setup depicted above. Even at a 30× baseline magnification our method produces sharper results. AbstractWe present Extreme View Synthesis, a solution for novel view extrapolation that works even when the number of input images is small-as few as two. In this context, occlusions and depth uncertainty are two of the most pressing issues, and worsen as the degree of extrapolation increases. We follow the traditional paradigm of performing depthbased warping and refinement, with a few key improvements. First, we estimate a depth probability volume, rather than just a single depth value for each pixel of the novel view. This allows us to leverage depth uncertainty in challenging regions, such as depth discontinuities. After using it to get an initial estimate of the novel view, we explicitly combine learned image priors and the depth uncertainty to synthesize a refined image with less artifacts. Our method is the first to show visually pleasing results for baseline magnifications of up to 30×. The code is available at https: /

Estimation of 3D motion in a dynamic scene from a temporal pair of images is a core task in many scene understanding problems. In real-world applications, a dynamic scene is commonly captured by a moving camera (i.e., panning, tilting or hand-held), increasing the task complexity because the scene is observed from di↵erent viewpoints. The primary challenge is the disambiguation of the camera motion from scene motion, which becomes more di cult as the amount of rigidity observed decreases, even with successful estimation of 2D image correspondences. Compared to other state-of-the-art 3D scene flow estimation methods, in this paper, we propose to learn the rigidity of a scene in a supervised manner from an extensive collection of dynamic scene data, and directly infer a rigidity mask from two sequential images with depths. With the learned network, we show how we can e↵ectively estimate camera motion and projected scene flow using computed 2D optical flow and the inferred rigidity mask. For training and testing the rigidity network, we also provide a new semi-synthetic dynamic scene dataset (synthetic foreground objects with a real background) and an evaluation split that accounts for the percentage of observed non-rigid pixels. Through our evaluation, we show the proposed framework outperforms current state-of-the-art scene flow estimation methods in challenging dynamic scenes.

No abstract

F inding ways to preserve cultural heritage and historic sites is an important problem. These sites are subject to erosion and vandalism, and as long-lived artifacts, they have gone through many phases of construction, damage, and repair. It's important to keep an accurate record of these sites' current conditions by using 3D model building technology, so preservationists can track changes, foresee structural problems, and allow a wider audience to virtually see and tour these sites. Due to the complexity of these sites, building 3D models is time consuming and difficult, usually involving much manual effort. Recently, the advent of new 3D range scanning devices has provided means to preserve these sites digitally and preserve the historic record by building accurate geometric and photorealistic 3D models. This data provides some exciting possibilities for creating models, but at the cost of scaling up existing methods to handle the extremely large point sets these devices create. This reinforces the need for automatic methods of registering, merging, and abstracting the dense range data sets. Other projects have addressed this and similar problems. 1-6 Each of these projects differs in the way they create models and in the amount of human interaction in the process. Our work centers on developing and automating new methods to recover complete geometric and photometric models of large sites. We're developing methods for data abstraction and compression through segmentation, 3D-to-3D registration (both coarse and fine), 2D-to-3D texture mapping of the models with imagery, and robotic automation of the sensing task. The methods we've developed are also suitable for a variety of other applications related to large-scale model building.

No abstract