Learning 3D structure from 2D images using LBP features

Abstract

An automatic machine learning strategy for computing the 3D structure of monocular images from a single image query using Local Binary Patterns is presented. The 3D structure is inferred through a training set composed by a repository of color and depth images, assuming that images with similar structure present similar depth maps. Local Binary Patterns are used to characterize the structure of the color images. The depth maps of those color images with a similar structure to the query image are adaptively combined and filtered to estimate the final depth map. Using public databases, promising results have been obtained outperforming other state-of-theart algorithms and with a computational cost similar to the most efficient 2D-to-3D algorithms.

Learning-based depth estimation from 2D images using GIST and saliency

“…We compute the average and median value of metric C obtained in the LOOCV test across all images in the Kinect- NYU dataset and compare the proposed approach with the Depth Transfer approach by Karsch et al [6], the HOG-based Depth Learning solution of Konrad et al [8] and the LBPBased Learning algorithm of Herrera et al [9]. The results are shown in Table 1, where, as can be observed, the proposed approach outperforms the results of the other state-of-the-art methods for both the average the median of the metric C. This improvement of the results is attributed to the use of the GIST features, and the saliency based weights used to select the k most similar images.…”

“…This approach is an extension of our previous work [9] with two main contributions. The first contribution is the use of a GIST-based descriptor as a representation of structure in color images.…”

“…They accomplished to reduce the processing time by several orders of magnitude primarily due to skipping the SIFT-flow alignment that turned out to bring minimal quality gains while incurring very high computational cost. A new approach, based on LBP features and using an adaptive number of similar images in the conversion was introduced by Herrera et al [9]. Since the computational cost of such methods is proportional to the size of the dataset used, the approach becomes impractical on very large datasets where the number of similar images to a query is very large.…”

Edge-based depth gradient refinement for 2D to 3D learned prior conversion

“…The difference with our previous approaches [6][7] is the learning based prior generations performed during the clustering and the edge-based post-processing stage that enhances the scene depth map estimation.…”

“…Also a Joint Bilateral Filter is used to enhance the resulting depth map. In our previous work [6], a new approach based on Local Binary Patterns (LBP) features are used to find an adaptive number of similar images that are fused in a weighted way to estimate the depth scene structure. Since the computational cost of these methods is proportional to the size of the database, these algorithms become impractical when using large databases.…”