OmniPhoto-360° VR photograph Figure 1: OmniPhotos are 360 • VR photographs with motion parallax that can be casually captured in a single 360 • video sweep. Capturing takes 3-10 seconds and, once processed into an image-based scene representation, OmniPhotos can be viewed freely in consumer VR headsets. Please note that our figures are animated to best convey our results; please view with Adobe Reader.

In recent years consumer-level depth cameras have been adopted in various applications. However, they often produce depth maps at a not very high fram-rate (around 30 frames per second), preventing them from being used for applications like digitizing human performance involving fast motion. On the other hand there are available low-cost high frame-rate video cameras. This motivates us to develop a hybrid camera that consists of a high frame-rate video camera and a low frame-rate depth camera, and to allow temporal interpolation of depth maps with the help of auxiliary color images. To achieve this we develop a novel algorithm that reconstructs intermediate depth maps and estimates scene flow simultaneously. We have tested our algorithm on various examples involving fast, non-rigid motions of single or multiple objects. Our experiments show that our scene flow estimation method is more precise than a purely tracking based method and the state-of-the-art techniques.

Virtual reality headsets are becoming increasingly popular, yet it remains difficult for casual users to capture immersive 360° VR panoramas. State-of-the-art approaches require capture times of usually far more than a minute and are often limited in their supported range of head motion. We introduce OmniPhotos, a novel approach for quickly and casually capturing high-quality 360° panoramas with motion parallax. Our approach requires a single sweep with a consumer 360° video camera as input, which takes less than 3 seconds to capture with a rotating selfie stick or 10 seconds handheld. This is the fastest capture time for any VR photography approach supporting motion parallax by an order of magnitude. We improve the visual rendering quality of our OmniPhotos by alleviating vertical distortion using a novel deformable proxy geometry, which we fit to a sparse 3D reconstruction of captured scenes. In addition, the 360° input views significantly expand the available viewing area, and thus the range of motion, compared to previous approaches. We have captured more than 50 OmniPhotos and show video results for a large variety of scenes. We will make our code available.

No abstract