Depth Map Super-Resolution Using Guided Deformable Convolution

Kim, Joon-Yeon; Ji, Seo-Won; Baek, Seung-Jin; Jung, Seung Won; Ko, Sung-Jea

doi:10.1109/access.2021.3076853

Cited by 3 publications

(1 citation statement)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Cao et al [20] propose a novel dual auto-encoder attention network (DAEANet) which includes two auto-encoder networks, where guidance auto-encoder network (GAENet) and target auto-encoder network (TAENet) aim to extract feature information from intensity image and depth image. Kim et al [21] propose a novel depth image super-resolution method using guided deformable convolution, which obtains 2D kernel offsets of the depth features from the guidance features to significantly alleviate the texture copying artifacts in the resultant depth image. Guo et al [22] propose a two-branch network to achieve depth image super-resolution with highresolution guidance image, which can be viewed as a prior to guide the low-resolution depth image to restore the missing high-frequency details of structures.…”

Section: Introductionmentioning

confidence: 99%

Improved Upsampling Based Depth Image Super-Resolution Reconstruction

Zhou

Wang

et al. 2023

IEEE Access

View full text Add to dashboard Cite

Constrained by current sensing technology, depth camera only acquires a low-resolution depth image that does not meet actual requirements. To solve this problem, this paper take a divide-and-conquer strategy to synthesize a high-resolution depth image from a low-resolution range image under the guidance of a registered high-resolution color image. Initially, the depth image is divided into planar areas and edge regions. For different zones, we exploit different methods to interpolate the missing depths. At planar area, the linear interpolation method is employed to perform upsampling. At edge region, a segmentation-separation upsampling method is used to interpolate the missing values. Then the upsampling result are refined on the Depth CNN that is built in this paper. We conduct extensive experiments on the benchmark database and real world data with various upsampling rates to illustrate the upsampling ability of our method. The comparison with classical super-resolution algorithms demonstrates that our upsampling algorithm achieves the best quality with fewer artifacts and our depth CNN outperforms the most state-of-the-art methods in terms of qualitative and quantitative evaluations.

show abstract

Section: Introductionmentioning

confidence: 99%

Improved Upsampling Based Depth Image Super-Resolution Reconstruction

Zhou

Wang

et al. 2023

IEEE Access

View full text Add to dashboard Cite

show abstract

Depth Map Super-Resolution via Cascaded Transformers Guidance

Ariav

Cohen

2022

Front. Signal Process.

View full text Add to dashboard Cite

Depth information captured by affordable depth sensors is characterized by low spatial resolution, which limits potential applications. Several methods have recently been proposed for guided super-resolution of depth maps using convolutional neural networks to overcome this limitation. In a guided super-resolution scheme, high-resolution depth maps are inferred from low-resolution ones with the additional guidance of a corresponding high-resolution intensity image. However, these methods are still prone to texture copying issues due to improper guidance by the intensity image. We propose a multi-scale residual deep network for depth map super-resolution. A cascaded transformer module incorporates high-resolution structural information from the intensity image into the depth upsampling process. The proposed cascaded transformer module achieves linear complexity in image resolution, making it applicable to high-resolution images. Extensive experiments demonstrate that the proposed method outperforms state-of-the-art techniques for guided depth super-resolution.

show abstract

Fully Cross-Attention Transformer for Guided Depth Super-Resolution

Ariav

Cohen

2023

Sensors

View full text Add to dashboard Cite

Modern depth sensors are often characterized by low spatial resolution, which hinders their use in real-world applications. However, the depth map in many scenarios is accompanied by a corresponding high-resolution color image. In light of this, learning-based methods have been extensively used for guided super-resolution of depth maps. A guided super-resolution scheme uses a corresponding high-resolution color image to infer high-resolution depth maps from low-resolution ones. Unfortunately, these methods still have texture copying problems due to improper guidance from color images. Specifically, in most existing methods, guidance from the color image is achieved by a naive concatenation of color and depth features. In this paper, we propose a fully transformer-based network for depth map super-resolution. A cascaded transformer module extracts deep features from a low-resolution depth. It incorporates a novel cross-attention mechanism to seamlessly and continuously guide the color image into the depth upsampling process. Using a window partitioning scheme, linear complexity in image resolution can be achieved, so it can be applied to high-resolution images. The proposed method of guided depth super-resolution outperforms other state-of-the-art methods through extensive experiments.

show abstract

Depth Map Super-Resolution Using Guided Deformable Convolution

Cited by 3 publications

References 36 publications

Improved Upsampling Based Depth Image Super-Resolution Reconstruction

Improved Upsampling Based Depth Image Super-Resolution Reconstruction

Depth Map Super-Resolution via Cascaded Transformers Guidance

Fully Cross-Attention Transformer for Guided Depth Super-Resolution

Contact Info

Product

Resources

About