Depth Super-Resolution via Joint Color-Guided Internal and External Regularizations

Liu, Xianming; Zhai, Deming; Chen, Rong; Ji, Xiangyang; Zhao, Debin; Gao, Wen

doi:10.1109/tip.2018.2875506

Cited by 59 publications

(41 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The works in [33,36] utilized a nonconvex penalty function to improve robustness of the smoothness regularization, which reduced the texture copy artifacts. Liu et al [23] proposed a gradient consistency regularizer to remedy the structure discrepancy problem, which can be viewed as a special form of edge image.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Deep edge map guided depth super resolution

Jiang

Yue

Lai

et al. 2021

Signal Processing: Image Communication

View full text Add to dashboard Cite

Accurate edge reconstruction is critical for depth map super resolution (SR). Therefore, many traditional SR methods utilize edge maps to guide depth SR. However, it is difficult to predict accurate edge maps from low resolution (L-R) depth maps. In this paper, we propose a deep edge map guided depth SR method, which includes an edge prediction subnetwork and an SR subnetwork. The edge prediction subnetwork takes advantage of the hierarchical representation of color and depth images to produce accurate edge maps, which promote the performance of SR subnetwork. The SR subnetwork is a disentangling cascaded network to progressively upsample SR result, where every level is made up of a weight sharing module and an adaptive module. The weight sharing module extracts the general features in different levels, while the adaptive module transfers the general features to the specific features to adapt to different degraded inputs. Quantitative and qualitative evaluations on various datasets with different magnification factors demonstrate the effectiveness and promising performance of the proposed method. In addition, we construct a benchmark dataset captured by Kinect-v2 to facilitate research on real-world depth map SR.

show abstract

Section: Related Workmentioning

confidence: 99%

“…How to synthesize satisfactory edge maps with RGB-D pairs is another problem. The extracted edges tend to be inaccurate or discontinuous when adopting simple edge detection methods [22,23,24] or when the external dataset is inad- equate [22,20,25] for sparse representation based and example based methods.…”

Section: Introductionmentioning

confidence: 99%

Deep edge map guided depth super resolution

Jiang

Yue

Lai

et al. 2021

Signal Processing: Image Communication

View full text Add to dashboard Cite

show abstract

“…The optimization-based methods exploit various optimization models for color guided SR, including MRF [28], [29], auto-regressive (AR) model [30], [31], weighted least squares (WLS) [32]- [34], total variation (TV) [35], [36], and graph signal model [37]. Specifically, Diebel et al [28] design a MRF framework, which contains a pairwise depth measurement potential and an image guided depth smoothness prior potential.…”

Section: ) Color Guided Depth Map Super-resolutionmentioning

confidence: 99%

“…Jiang et al [36] propose a unified depth SR model with transform domain regularization and spatial multi-directional TV prior. Liu et al [37] design a depth SR optimization framework by combining both internal graph-signal smoothness prior and external depth-color gradient consistency. Yu et al [38] propose color guided depth up-sampling based on edge sparsity and super-weighted L 0 gradient minimization.…”

Section: ) Color Guided Depth Map Super-resolutionmentioning

confidence: 99%

Guided Depth Map Super-Resolution Using Recumbent Y Network

Dong

Lin

2020

IEEE Access

View full text Add to dashboard Cite

Low spatial resolution is a well-known problem for depth maps captured by low-cost consumer depth cameras. Depth map super-resolution (SR) can be used to enhance the resolution and improve the quality of depth maps. In this paper, we propose a recumbent Y network (RYNet) to integrate the depth information and intensity information for depth map SR. Specifically, we introduce two weight-shared encoders to respectively learn multi-scale depth and intensity features, and a single decoder to gradually fuse depth information and intensity information for reconstruction. We also design a residual channel attention based atrous spatial pyramid pooling structure to further enrich the feature's scale diversity and exploit the correlations between multi-scale feature channels. Furthermore, the violations of co-occurrence assumption between depth discontinuities and intensity edges will generate texture-transfer and depth-bleeding artifacts. Thus, we propose a spatial attention mechanism to mitigate the artifacts by adaptively learning the spatial relevance between intensity features and depth features and reweighting the intensity features before fusion. Experimental results demonstrate the superiority of the proposed RYNet over several state-of-the-art depth map SR methods. INDEX TERMS Depth map super-resolution, convolutional neural network, UNet network, atrous spatial pyramid pooling, attention mechanism.

show abstract

“…To improve the processing performance of image texture-free regions, Fan et al [22] presented a shape focusing method combined with a 3D adjustable filter that considered edge response and image blurring. Liu et al [23] proposed a graph Laplacian regularizer to preserve the inherent piecewise smoothness of depth, and this method demonstrated effective filtering. An iterative algorithm that combines stationary wavelet transform, bilateral filtering, Bayesian estimation and anisotropic diffusion filtering was used to reduce speckle noise in SAR images [24].…”

mentioning

confidence: 99%

Volumetric Tooth Wear Measurement of Scraper Conveyor Sprocket Using Shape from Focus-Based Method

Ding

Liu

2019

Applied Sciences

View full text Add to dashboard Cite

Volumetric tooth wear measurement is important to assess the life of scraper conveyor sprocket. A shape from focus-based method is used to measure scraper conveyor sprocket tooth wear. This method reduces the complexity of the process and improves the accuracy and efficiency of existing methods. A prototype set of sequence images taken by the camera facing the sprocket teeth is collected by controlling the fabricated track movement. In this method, a normal distribution operator image filtering is employed to improve the accuracy of an evaluation function value calculation. In order to detect noisy pixels, a normal operator is used, which involves with using a median filter to retain as much of the original image information as possible. In addition, an adaptive evaluation window selection method is proposed to address the difficulty associated with identifying an appropriate evaluation window to calculate the focused evaluation value. The shape and size of the evaluation window are autonomously determined using the correlation value of the grey scale co-occurrence matrix generated from the measured pixels' neighbourhood pixels. A reverse engineering technique is used to quantitatively verify the shape volume recovery accuracy of different evaluation windows. The test results demonstrate that the proposed method can effectively measure sprocket teeth wear volume with an accuracy up to 97.23%. discussed the wear condition of a driving sprocket and the influence of wear on the sliding distance by taking the sliding speed and sliding distance of the meshing process as the index. Wang et al. [8] also analyzed the relationship between the deformation of a ring chain and driving sprocket wear by combining numerical analysis with experiments. However, these methods are not only tedious and time-consuming, they are also not sufficiently accurate or efficient.The research of computer vision in industry field has attracted more and more attention of many researchers. Alverdi et al. [9] proposed a new way of using images to model the kerf profile in abrasive water jet milling. Qian et al. [10] presented an algorithm to compute the axis and generatrix focus on complex surfaces or irregular surfaces. A new monitoring technique for burr detection was proposed for the optimization of drill geometry and process parameters [11]. In addition, as a relatively simple and practical 3D reconstruction technology, shape from focus (SFF) has been applied to tool wear measurements [12,13], LCD/TFT (Liquid Crystal Display/Thin-Film Technology) display manufacturing [14] and grinding wheel surface morphology [15], etc.To realize 3D surface topography restoration, in 1988, Darrell et al. [16] proposed using a Laplace operator-Gauss fitting method to search the clear frame of pixels in the sequence partial focus image according to image focusing information. In the 1990s, Nayar et al. [17,18] proposed an SFF-based method and obtained the height information of the corresponding surface of the window image by searching the image position cor...

show abstract

Depth Super-Resolution via Joint Color-Guided Internal and External Regularizations

Cited by 59 publications

References 21 publications

Deep edge map guided depth super resolution

Deep edge map guided depth super resolution

Guided Depth Map Super-Resolution Using Recumbent Y Network

Volumetric Tooth Wear Measurement of Scraper Conveyor Sprocket Using Shape from Focus-Based Method

Contact Info

Product

Resources

About