A 3D Imaging Framework Based on High-Resolution Photometric-Stereo and Low-Resolution Depth

Zheng, Lu; Tai, Yu-Wing; Deng, Fanbo; Ben-Ezra, Moshe; Brown, Michael S.

doi:10.1007/s11263-012-0589-5

Cited by 15 publications

(6 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In a similar vein as ours, Lu et al [18] combined a low resolution depth with high resolution photometric stereo where the depth map is obtained via structured light. Compared with the structured light results that are highly noisy, the morphable 3D face geometry is smoother but less accurate.…”

Section: D-to-3d Conversionmentioning

confidence: 98%

Sparse Photometric 3D Face Reconstruction Guided by Morphable Models

Cao

Chen

et al. 2018

2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition

View full text Add to dashboard Cite

Figure 1: Sample results using our sparse PS reconstruction. By using just 5 input images (left), our method can recover very high quality 3D face geometry with fine geometric details. AbstractWe present a novel 3D face reconstruction technique that leverages sparse photometric stereo (PS) and latest advances on face registration/modeling from a single image. We observe that 3D morphable faces approach[15] provides a reasonable geometry proxy for light position calibration. Specifically, we develop a robust optimization technique that can calibrate per-pixel lighting direction and illumination at a very high precision without assuming uniform surface albedos. Next, we apply semantic segmentation on input images and the geometry proxy to refine hairy vs. bare skin regions using tailored filters. Experiments on synthetic and real data show that by using a very small set of images, our technique is able to reconstruct fine geometric details such as wrinkles, eyebrows, whelks, pores, etc, comparable to and sometimes surpassing movie quality productions.

show abstract

Section: D-to-3d Conversionmentioning

confidence: 98%

Sparse Photometric 3D Face Reconstruction Guided by Morphable Models

Cao

Chen

et al. 2018

2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition

View full text Add to dashboard Cite

show abstract

“…Several methods were proposed to coalign the depth edges in the super-resolved depth map with edges in the high-resolution color image [2], [58], [59], [60], [61], [62], but such approaches only consider sparse color features and may thus miss thin geometric structures. Some authors super-resolve the photometric stereo results [63], and others generate high-resolution images using photometric stereo [64], but none employ low-resolution depth clues except those of [65], who combine calibrated photometric stereo with structured light sensing. However, this involves a non-standard setup and careful lighting calibration, and reflectance is assumed to be uniform.…”

Section: Related Workmentioning

confidence: 99%

Photometric Depth Super-Resolution

Haefner

Peng

Verma

et al. 2020

IEEE Trans. Pattern Anal. Mach. Intell.

View full text Add to dashboard Cite

This study explores the use of photometric techniques (shape-from-shading and uncalibrated photometric stereo) for upsampling the low-resolution depth map from an RGB-D sensor to the higher resolution of the companion RGB image. A single-shot variational approach is first put forward, which is effective as long as the target's reflectance is piecewise-constant. It is then shown that this dependency upon a specific reflectance model can be relaxed by focusing on a specific class of objects (e.g., faces), and delegate reflectance estimation to a deep neural network. A multi-shot strategy based on randomly varying lighting conditions is eventually discussed. It requires no training or prior on the reflectance, yet this comes at the price of a dedicated acquisition setup. Both quantitative and qualitative evaluations illustrate the effectiveness of the proposed methods on synthetic and real-world scenarios.

show abstract

“…Another variant is to train a network end-toend to directly learn a mapping between low-resolution and high-resolution depth images [51]. Recently, it was shown that taking into account photometric approaches helps solving the problem of depth super-resolution, due to incorporating the physical relationship between intensity and depth [19,21,38,44]. Our approach builds upon these ideas to recover a fine scale, geometrically detailed depth image based on a methodology called photometric stereo.…”

Section: Related Workmentioning

confidence: 99%

Inferring Super-Resolution Depth from a Moving Light-Source Enhanced RGB-D Sensor: A Variational Approach

Sang¹,

Haefner²,

Cremers³

2019

Preprint

View full text Add to dashboard Cite

A novel approach towards depth map super-resolution using multi-view uncalibrated photometric stereo is presented. Practically, an LED light source is attached to a commodity RGB-D sensor and is used to capture objects from multiple viewpoints with unknown motion. This nonstatic camera-to-object setup is described with a nonconvex variational approach such that no calibration on lighting or camera motion is required due to the formulation of an end-to-end joint optimization problem. Solving the proposed variational model results in high resolution depth, reflectance and camera pose estimates, as we show on challenging synthetic and real-world datasets.

show abstract

A 3D Imaging Framework Based on High-Resolution Photometric-Stereo and Low-Resolution Depth

Cited by 15 publications

References 18 publications

Sparse Photometric 3D Face Reconstruction Guided by Morphable Models

Sparse Photometric 3D Face Reconstruction Guided by Morphable Models

Photometric Depth Super-Resolution

Inferring Super-Resolution Depth from a Moving Light-Source Enhanced RGB-D Sensor: A Variational Approach

Contact Info

Product

Resources

About