Robust Color Guided Depth Map Restoration

Abstract

One of the most challenging issues in color guided depth map restoration is the inconsistency between color edges in guidance color images and depth discontinuities on depth maps. This makes the restored depth map suffer from texture copy artifacts and blurring depth discontinuities. To handle this problem, most state-of-the-art methods design complex guidance weight based on guidance color images and heuristically make use of the bicubic interpolation of the input depth map. In this paper, we show that using bicubic interpolated depth map can blur depth discontinuities when the upsampling factor is large and the input depth map contains large holes and heavy noise. In contrast, we propose a robust optimization framework for color guided depth map restoration. By adopting a robust penalty function to model the smoothness term of our model, we show that the proposed method is robust against the inconsistency between color edges and depth discontinuities even when we use simple guidance weight. To the best of our knowledge, we are the first to solve this problem with a principled mathematical formulation rather than previous heuristic weighting schemes. The proposed robust method performs well in suppressing texture copy artifacts. Moreover, it can better preserve sharp depth discontinuities than previous heuristic weighting schemes. Through comprehensive experiments on both simulated data and real data, we show promising performance of the proposed method.

Semi-Global Weighted Least Squares in Image Filtering

“…9. The main challenges of guided depth upsampling are texture copy artifacts and blurring edges [18]. As illustrated in highlighted regions, our SG-WLS shows better performance in handing the challenges than compared methods.…”

“…Note that for r = 1, only Eqs. (9), (12), (14), (15), (17), (18) and (20) are needed. For r = 2, only Eq.…”

Deep Convolutional Neural Network for Multi-Modal Image Restoration and Fusion

“…(6) Since c k is fixed, we can further simplify (6) by denotinĝ x = x− k d c k * c k , which leads to the following expression:…”

“…M ULTI-MODAL image processing has been attracting increasing interest from the computer vision community, due to a variety of intriguing applications, e.g., image style transfer [1], [2], image fusion [3], [4], RGB guided depth image super-resolution [5], [6], image denoising [7]. Based on the reconstruction target, these applications can be roughly classified into two categories, the multi-modal image restoration (MIR) and multi-modal image fusion (MIF) task.…”

Fast Color-guided Depth Denoising for RGB-D Images by Graph Filtering

“…Color-guided methods for depth image denoising in [6]- [8] formulate the problem as an optimization problem and utilize the guidelines from the color images by modifying the regularization. However, these methods are likely to be intractable and time-consuming to implement.…”