Learning Single-Image Depth From Videos Using Quality Assessment Networks

Abstract

Depth estimation from a single image in the wild remains a challenging problem. One main obstacle is the lack of high-quality training data for images in the wild. In this paper we propose a method to automatically generate such data through Structure-from-Motion (SfM) on Internet videos. The core of this method is a Quality Assessment Network that identifies high-quality reconstructions obtained from SfM. Using this method, we collect singleview depth training data from a large number of YouTube videos and construct a new dataset called YouTube3D. Experiments show that YouTube3D is useful in training depth estimation networks and advances the state of the art of single-view depth estimation in the wild. Project website: https

Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-Shot Cross-Dataset Transfer

“…they are frozen in action while the camera moves through the scene). Chen et al [39] propose an approach to automatically assess the quality of sparse SfM reconstructions in order to construct a large dataset. Wang et al [33] build a large dataset from stereo videos sourced from the web, while Cho et al [40] collect a dataset of outdoor scenes with handheld stereo cameras.…”

Ranking-Based Salient Object Detection and Depth Prediction for Shallow Depth-of-Field

“…Quantitative and qualitative results: In Table 4 , we compare our MDP model with 7 state-of-the-art methods, including DIW [ 41 ], DL [ 5 ], RW [ 15 ], MD [ 42 ], Y3D [ 44 ], MC [ 43 ], and HRWSI [ 14 ]. For the definition of the metrics, please refer to [ 14 ].…”

“…In other words, these methods trained on one dataset often fail to get promising predictions on a different one. To learn depth in general scenes with a single model, recent studies [ 15 , 41 , 42 , 43 , 44 ] start from constructing in-the-wild RGB-D datasets. For example, Chen et al [ 41 ] propose the DIW dataset, which consists of about 495K natural images.…”

Predicting 3D shapes, masks, and properties of materials inside transparent containers, using the TransProteus CGI dataset

“…Extracting a 3D model from photos could be achieved using a single image or multiple images covering different viewpoints. 11–16,34–36 The standard depth and 3D prediction methods assume that light moves in straight lines, 10,37 an assumption that fails with transparent objects. 1–3 As a result, standard methods for extracting the 3D models from images like stereo matching and structured light fail on transparent objects.…”