Non-Line-of-Sight Surface Reconstruction Using the Directional Light-Cone Transform

Young, Sean I.; Lindell, David B.; Girod, Bernd; Taubman, David; Wetzstein, Gordon

doi:10.1109/cvpr42600.2020.00148

Cited by 40 publications

(29 citation statements)

References 43 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Plenty of methods in reconstructing the hidden objects from transient measurement have been developed [1,39,20,12,16,18,27,32,38,41,11,29,22,24,23,45]. As a precursory work in this field, Velten et al [39] propose a filtered back-projection method to recover the hidden ob-jects from transient measurements.…”

Section: Nlos Reconstruction Methodsmentioning

confidence: 99%

Towards Non-Line-of-Sight Photography

Peng¹,

Mu²,

Nam³

et al. 2021

Preprint

View full text Add to dashboard Cite

Non-line-of-sight (NLOS) imaging is based on capturing the multi-bounce indirect reflections from the hidden objects. Active NLOS imaging systems rely on the capture of the time of flight of light through the scene, and have shown great promise for the accurate and robust reconstruction of hidden scenes without the need for specialized scene setups and prior assumptions. Despite that existing methods can reconstruct 3D geometries of the hidden scene with excellent depth resolution, accurately recovering object textures and appearance with high lateral resolution remains an challenging problem. In this work, we propose a new problem formulation, called NLOS photography, to specifically address this deficiency. Rather than performing an intermediate estimate of the 3D scene geometry, our method follows a data-driven approach and directly reconstructs 2D images of a NLOS scene that closely resemble the pictures taken with a conventional camera from the location of the relay wall. This formulation largely simplifies the challenging reconstruction problem by bypassing the explicit modeling of 3D geometry, and enables the learning of a deep model with a relatively small training dataset. The results are NLOS reconstructions of unprecedented lateral resolution and image quality.

show abstract

Section: Nlos Reconstruction Methodsmentioning

confidence: 99%

Towards Non-Line-of-Sight Photography

Peng¹,

Mu²,

Nam³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Deep-inverse correlography [14] Deep matrix factorization [67] Physics-based DeepRL framework [40] 3D reconstruction Learned feature embeddings [13,47] Conventional methods Modulated light source + TOF camera [3,46] Active light source + Interferometer [30,44,45] Laser + Conventional camera [15,48,[49][50][51] ToF-based [1,3,11,24,27] Wave-based [5] Phasor field [4,10,45] Backprojection [1,2,26] Inverse methods [3,11,21,22] Surface normal algorithms [24,30,33,62] Wave-based / Phasor field [4,5,10,45] Conventional camera + Ambient light [6,67,[73][74][75][76][77] Conventional camera + Incoherent source [8,68] Interferometer + Incoherent...…”

Section: Pose Estimationmentioning

confidence: 99%

“…( 7), the surface normal reconstruction and volumetric albedo reconstruction can be combined to achieve good results [24]. Similarly, fusing the surface reconstruction into the convolution kernel of the LCT can also achieve better reconstruction results than using only the volumetric albedo representation [33].…”

Section: Reconstruction Algorithmsmentioning

confidence: 99%

See 1 more Smart Citation

Recent Advances on Non-Line-of-Sight Imaging: Conventional Physical Models, Deep Learning, and New Scenes

Geng¹,

Yang²,

Chen³

2021

Preprint

View full text Add to dashboard Cite

As an emerging technology that has attracted huge attention, non-line-of-sight (NLOS) imaging can reconstruct hidden objects by analyzing the diffuse reflection on a relay surface, with broad application prospects in the fields of autonomous driving, medical imaging, and defense. Despite the challenges of low signal-to-noise ratio (SNR) and high ill-posedness, NLOS imaging has been developed rapidly in recent years. Most current NLOS imaging technologies use conventional physical models, constructing imaging models through active or passive illumination and using reconstruction algorithms to restore hidden scenes. Moreover, deep learning algorithms for NLOS imaging have also received much attention recently. This paper presents a comprehensive overview of both conventional and deep learningbased NLOS imaging techniques. Besides, we also survey new proposed NLOS scenes, and discuss the challenges and prospects of existing technologies. Such a survey can help readers have an overview of different types of NLOS imaging, thus expediting the development of seeing around corners.

show abstract

“…To make the problem tractable, the pioneering backprojection (BP) technique and its variations assume smooth objects so that scene recovery can be modeled as deconvolution [1], [2], [8], [31]. Alternatively, the light-cone transform (LCT) based methods collocate the illumination and sensing spots on the relay wall so that the forward imaging model can be simplified as 3D convolution [4], [15], [34], [35] where advanced signal processing techniques such as Wiener filters [34], [35], [38] can further reduce noise. Assuming the scene is near diffuse, analysis-by-synthesis algorithms can improve reconstruction [18], [36].…”

Section: Introductionmentioning

confidence: 99%

Non-line-of-Sight Imaging via Neural Transient Fields

Shen,

Wang,

Liu

et al. 2021

Preprint

View full text Add to dashboard Cite

We present a neural modeling framework for Non-Line-of-Sight (NLOS) imaging. Previous solutions have sought to explicitly recover the 3D geometry (e.g., as point clouds) or voxel density (e.g., within a pre-defined volume) of the hidden scene. In contrast, inspired by the recent Neural Radiance Field (NeRF) approach, we use a multi-layer perceptron (MLP) to represent the neural transient field or NeTF. However, NeTF measures the transient over spherical wavefronts rather than the radiance along lines. We therefore formulate a spherical volume NeTF reconstruction pipeline, applicable to both confocal and non-confocal setups. Compared with NeRF, NeTF samples a much sparser set of viewpoints (scanning spots) and the sampling is highly uneven. We thus introduce a Monte Carlo technique to improve the robustness in the reconstruction. Comprehensive experiments on synthetic and real datasets demonstrate NeTF provides higher quality reconstruction and preserves fine details largely missing in the state-of-the-art.

show abstract

Non-Line-of-Sight Surface Reconstruction Using the Directional Light-Cone Transform

Cited by 40 publications

References 43 publications

Towards Non-Line-of-Sight Photography

Towards Non-Line-of-Sight Photography

Recent Advances on Non-Line-of-Sight Imaging: Conventional Physical Models, Deep Learning, and New Scenes

Non-line-of-Sight Imaging via Neural Transient Fields

Contact Info

Product

Resources

About