Color-Guided Depth Recovery From RGB-D Data Using an Adaptive Autoregressive Model

Yang, Jingyu; Ye, Xinchen; Li, Kun; Hou, Chunping; Wang, Yao

doi:10.1109/tip.2014.2329776

Cited by 292 publications

(237 citation statements)

References 37 publications

Supporting

Mentioning

237

Contrasting

Order By: Relevance

“…They formulate the energy functional with an anisotropic Total Generalized Variation prior, which is weighted by the gradients in the guidance intensity image. Yang et al [40] formulate the depth upsampling as a minimization of an adaptive color-guided auto-regressive model. One of the few learning based approaches is proposed by Kwon et al [23].…”

Section: Related Workmentioning

confidence: 99%

A Deep Primal-Dual Network for Guided Depth Super-Resolution

Riegler¹,

Ferstl²,

Rüther³

et al. 2016

Procedings of the British Machine Vision Conference 2016

View full text Add to dashboard Cite

In this paper we present a novel method to increase the spatial resolution of depth images. We combine a deep fully convolutional network with a non-local variational method in a deep primal-dual network. The joint network computes a noise-free, highresolution estimate from a noisy, low-resolution input depth map. Additionally, a highresolution intensity image is used to guide the reconstruction in the network. By unrolling the optimization steps of a first-order primal-dual algorithm and formulating it as a network, we can train our joint method end-to-end. This not only enables us to learn the weights of the fully convolutional network, but also to optimize all parameters of the variational method and its optimization procedure. The training of such a deep network requires a large dataset for supervision. Therefore, we generate high-quality depth maps and corresponding color images with a physically based renderer. In an exhaustive evaluation we show that our method outperforms the state-of-the-art on multiple benchmarks.

show abstract

Section: Related Workmentioning

confidence: 99%

A Deep Primal-Dual Network for Guided Depth Super-Resolution

Riegler¹,

Ferstl²,

Rüther³

et al. 2016

Procedings of the British Machine Vision Conference 2016

View full text Add to dashboard Cite

show abstract

“…For instance, Tosic et al [105] and Yang et al [117] provide methods to inpaint non-measured depth values due to certain causes (e.g. specular and absorptive surfaces) by respectively utilizing Kinect IR and RGB images.…”

Section: Introductionmentioning

confidence: 99%

Optimal 6D Object Pose Estimation with Commodity Depth Sensors

Landau

View full text Add to dashboard Cite

Accurate 6D object pose estimation, as well as other model-based shape matching objectives such as object detection, classification, and shape inspection, is prominent and necessary in a large number of domains and applications. This includes household and robotics applications to automate various tasks, where input depth measurements are aligned to a corresponding model of the object, such as a CAD that is composed of several interconnected parts, depending on the model's level of detail and complexity. Industrial metrology systems also compare the shapes of engineered components and equipment to their corresponding 3D models for quality assessment and anomaly detection; this often requires very costly and high resolution projection and detection mechanisms. Structuredlight laser scanners have, however, emerged as a viable option to provide cheap and reasonably accurate depth estimates, which have recently become commodity-priced within this decade due to the maturity of the technology. These sensors are also robust for semi-transparent and reflective surfaces, dynamic scenery, ambient background light, and temperature drift. Certain structured-light light coding methods unfortunately need to implement a complicated, non-invertible transformation of a distorted light pattern to generate the pixel-by-pixel depth estimates, thereby leading to a loss of information. This can result in poorly estimated or even missing depth data, especially when the object has small 3D features or non-ideal surface properties. Moreover, the informative rigid body constraint is blurred when the nonlinear 2D depth image to 3D pseudo-measurement point cloud transformation is applied, which is the domain where most model-based shape matching methods operate.Alternatively, estimating pose from information early in the structured-light sensing/processing chain has the potential to alleviate errors induced in subsequent nonlinear processing steps that are in fact locally scene dependent. This motivates one of the main contributions from this dissertation, which presents an asymptotically optimal maximum likelihood estimation method that operates directly on the raw output IR images. This is made possible by an extensive study on a commonly utilized commodity-priced structured-light sensor, which contributed to the proposed high-fidelity IR and depth image predictor and simulator that models the physics of the transmit and receive optics,

show abstract

“…Recently, the low-cost structured-light RGB-D cameras have been used to capture high-resolution color images and low-resolution depth maps [10]. Thus, depth map upsampling [11,12] followed by its enhancement [13] becomes an inevitable task because the quality of the DIBR process heavily depends on the accuracy of depth information. To improve the depth map of RGB-D cameras, the following problems should be solved.…”

Section: Introductionmentioning

confidence: 99%

Robust depth enhancement and optimization based on advanced multilateral filters

Chang

Chou

Yang

2017

EURASIP J. Adv. Signal Process.

View full text Add to dashboard Cite

Stereo matching of two distanced cameras and structured-light RGB-D cameras are the two common ways to capture the depth map, which conveys the per-pixel depth information of the image. However, the results with mismatched and occluded pixels would not provide accurately well-matched depth and image information. The mismatched depth-image relations would degrade the performances of view syntheses seriously in modern-day three-dimension video applications. Therefore, how to effectively utilize the image and depth to enhance themselves becomes more and more important. In this paper, we propose an advanced multilateral filter (AMF), which refers spatial, range, depth, and credibility information to achieve their enhancements. The AMF enhancements could sharpen the image, suppress noisy depth, filling depth holes, and sharpen the depth edges simultaneously. Experimental results demonstrate that the proposed method provides a superior performance, especially around the object boundary.

show abstract

Color-Guided Depth Recovery From RGB-D Data Using an Adaptive Autoregressive Model

Cited by 292 publications

References 37 publications

A Deep Primal-Dual Network for Guided Depth Super-Resolution

A Deep Primal-Dual Network for Guided Depth Super-Resolution

Optimal 6D Object Pose Estimation with Commodity Depth Sensors

Robust depth enhancement and optimization based on advanced multilateral filters

Contact Info

Product

Resources

About