Fight Ill-Posedness with Ill-Posedness: Single-shot Variational Depth Super-Resolution from Shading

Haefner, Bjoern; Quéau, Yvain; Möllenhoff, Thomas; Cremers, Daniel

doi:10.1109/cvpr.2018.00025

Cited by 40 publications

(29 citation statements)

References 61 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, this involves a non-standard setup and careful lighting calibration, and reflectance is assumed to be uniform. Such issues are circumvented in the building blocks [66] and [67] of this study, which deal with photometric depth super-resolution based on, respectively, shape-from-shading and photometric stereo. Let us present the former approach, which is a singleshot solution to photometric depth super-resolution based on a variational approach to shape-from-shading.…”

Section: Related Workmentioning

confidence: 99%

Photometric Depth Super-Resolution

Haefner

Peng

Verma

et al. 2020

IEEE Trans. Pattern Anal. Mach. Intell.

Self Cite

View full text Add to dashboard Cite

This study explores the use of photometric techniques (shape-from-shading and uncalibrated photometric stereo) for upsampling the low-resolution depth map from an RGB-D sensor to the higher resolution of the companion RGB image. A single-shot variational approach is first put forward, which is effective as long as the target's reflectance is piecewise-constant. It is then shown that this dependency upon a specific reflectance model can be relaxed by focusing on a specific class of objects (e.g., faces), and delegate reflectance estimation to a deep neural network. A multi-shot strategy based on randomly varying lighting conditions is eventually discussed. It requires no training or prior on the reflectance, yet this comes at the price of a dedicated acquisition setup. Both quantitative and qualitative evaluations illustrate the effectiveness of the proposed methods on synthetic and real-world scenarios.

show abstract

Section: Related Workmentioning

confidence: 99%

Photometric Depth Super-Resolution

Haefner

Peng

Verma

et al. 2020

IEEE Trans. Pattern Anal. Mach. Intell.

Self Cite

View full text Add to dashboard Cite

show abstract

“…In the absence of such priors, SfS can be combined with another 3D-reconstruction technique: the latter provides a coarse prior on geometry, whose details are then refined using SfS. In this view, SfS has been combined with shape-from-texture [123], structure-from-motion [38], multiview stereopsis [61,68,72], or depth sensors [41,85]. An alternative strategy to resolve the ambiguities of SfS consists in using additional images taken under varying lighting.…”

Section: Applications Of Sfsmentioning

confidence: 99%

A Comprehensive Introduction to Photometric 3D-Reconstruction

Durou

Falcone

Quéau

et al. 2020

Advances in Computer Vision and Pattern Recognition

View full text Add to dashboard Cite

show abstract

“…The model we used for evaluation was the same as the model tested on datasets A, B, and C without fine-tuning, and the evaluation criterion was still the RMSE. We mainly tested our method at the upscaling factors of 4 and 8, in comparison with methods from References [23,26,[39][40][41]. Our method produced the best performance on the image from the Middlebury dataset and performed nearly 20% better than the sub-optimal result (see Table 4).…”

Section: Evaluation Of Generalizationmentioning

confidence: 99%

Color-Guided Depth Map Super-Resolution Using a Dual-Branch Multi-Scale Residual Network with Channel Interaction

Chen

Gao

2020

Sensors

View full text Add to dashboard Cite

We designed an end-to-end dual-branch residual network architecture that inputs a low-resolution (LR) depth map and a corresponding high-resolution (HR) color image separately into the two branches, and outputs an HR depth map through a multi-scale, channel-wise feature extraction, interaction, and upsampling. Each branch of this network contains several residual levels at different scales, and each level comprises multiple residual groups composed of several residual blocks. A short-skip connection in every residual block and a long-skip connection in each residual group or level allow for low-frequency information to be bypassed while the main network focuses on learning high-frequency information. High-frequency information learned by each residual block in the color image branch is input into the corresponding residual block in the depth map branch, and this kind of channel-wise feature supplement and fusion can not only help the depth map branch to alleviate blur in details like edges, but also introduce some depth artifacts to feature maps. To avoid the above introduced artifacts, the channel interaction fuses the feature maps using weights referring to the channel attention mechanism. The parallel multi-scale network architecture with channel interaction for feature guidance is the main contribution of our work and experiments show that our proposed method had a better performance in terms of accuracy compared with other methods.

show abstract

Fight Ill-Posedness with Ill-Posedness: Single-shot Variational Depth Super-Resolution from Shading

Cited by 40 publications

References 61 publications

Photometric Depth Super-Resolution

Photometric Depth Super-Resolution

A Comprehensive Introduction to Photometric 3D-Reconstruction

Color-Guided Depth Map Super-Resolution Using a Dual-Branch Multi-Scale Residual Network with Channel Interaction

Contact Info

Product

Resources

About