Intensity Guided Depth Upsampling Using Edge Sparsity and Super-Weighted $L_0$ Gradient Minimization

Abstract

Although depth cameras acquire depth in dynamic scenes, the captured depth images are often noisy and of low resolution. Depth images have the physical nature of being represented by smooth regions and edges in between them, i.e. depth images have high edge sparsity in the gradient domain. In this paper, we propose intensity guided depth upsampling using edge sparsity and super-weighted L 0 gradient minimization. First, we get mutual structure between intensity and depth using joint mutual structure filtering. Second, we generate an initial depth image using recursive interpolation. Next, we generate weights for L 0 gradient minimization based on gradient and entropy of the intensity image, and upsample the depth image using super-weighted L 0 gradient minimization. In super-weighted L 0 gradient minimization, we combine two main terms: Hybrid data fidelity and weighted L 0 gradient regularization. The hybrid data fidelity term combines both zero-order and first-order data differences to suppress staircase artifacts, while the weighted L 0 gradient regularization term preserves depth structures and removes noise. Finally, we further refine the depth image using adaptive fast weighted median filtering. Experiments on Middlebury and realworld scene datasets verify that the proposed method produces edge preserving depth upsampling results and outperforms state-of-the-arts in terms of both visual quality and quantitative measurements. INDEX TERMS Depth upsampling, edge sparsity, intensity guided, L 0 gradient minimization, recursive interpolation, mutual structure, weighted median filtering.

Guided Depth Map Super-Resolution Using Recumbent Y Network

“…Liu et al [37] design a depth SR optimization framework by combining both internal graph-signal smoothness prior and external depth-color gradient consistency. Yu et al [38] propose color guided depth up-sampling based on edge sparsity and super-weighted L 0 gradient minimization.…”

Structure-preserving image smoothing with semantic cues

“…For qualitative and quantitative comparisons, we select these methods including: Bilinear interpolation (as a baseline), Liu et al [24] and Jung et al [49]. As shown in Fig.…”