Reconstructing rooms using photon echoes: A plane based model and reconstruction algorithm for looking around the corner

Abstract

Can we reconstruct the entire internal shape of a room if all we can directly observe is a small portion of one internal wall, presumably through a window in the room? While conventional wisdom may indicate that this is not possible, motivated by recent work on 'looking around corners', we show that one can exploit light echoes to reconstruct the internal shape of hidden rooms.Existing techniques for looking around the corner using transient images model the hidden volume using voxels and try to explain the captured transient response as the sum of the transient responses obtained from individual voxels. Such a technique inherently suffers from challenges with regards to low signal to background ratios (SBR) and has difficulty scaling to larger volumes. In contrast, in this paper, we argue for using a plane-based model for the hidden surfaces. We demonstrate that such a plane-based model results in much higher SBR while simultaneously being amenable to larger spatial scales. We build an experimental prototype composed of a pulsed laser source and a singlephoton avalanche detector (SPAD) that can achieve a time resolution of about 30ps and demonstrate high-fidelity reconstructions both of individual planes in a hidden volume and for reconstructing entire polygonal rooms composed of multiple planar walls.

Non-line-of-sight Reconstruction Using Efficient Transient Rendering

“…A smaller but more diverse group of work relies on the use of forward models to arrive at a scene hypothesis that best agrees with the measured data. Here, reported approaches range from combinatorial labeling schemes [Kirmani et al 2009] via frequency-domain inverse filtering (if the capture geometry is sufficiently constrained) [O'Toole et al 2018a] to variational methods using simple linearized light transport tensors [Heide et al 2014;Naik et al 2011] and simplistic models based on radiative transfer [Klein et al 2016;Pediredla et al 2017] that are (in principle) capable of expressing opacity effects like shadowing and occlusion, and physically plausible shading and that are closest to our proposed method. In concurrent work, Heide et al [2017] added such extra factors as additional weights into their least-squares data term, achieving non-line-of-sight reconstructions of significantly improved robustness.…”

Ellipsoidal path connections for time-gated rendering

“…These include streak cameras [Velten et al 2013], intensified CCD (ICCD) sensors [Cester et al 2019;Pediredla et al 2017b], single-photon avalanche diodes (SPADs) [Gariepy et al 2015], continuous-wave time-of-flight sensors [Heide et al 2013;Peters et al 2015], Kerr gates [Ham 2019;Schmidt et al 2003;Takahashi et al 1994;Zhan et al 2016], and interferometry [Gkioulekas et al 2015]. Transient imaging has recently attracted increased interest within computer graphics, for applications such as non-line-of-sight imaging [Buttafava et al 2015;Chan et al 2017;Kirmani et al 2009;O'Toole et al 2018;Pediredla et al 2017aPediredla et al , 2019aTsai et al 2017;Velten et al 2012;Xin et al 2019], separating light transport components Wu et al 2014a,b], and inverting and seeing through scattering [Gkioulekas et al 2016;Satat et al 2016].…”

Computational periscopy with an ordinary digital camera

“…In addition to lowering the cost of NLOS imaging systems, SPAD-based systems have facilitated the extension of previous round-trip distances of around 1 m to a few meters for NLOS hidden-object estimation 5 and over 50 m for human localization at long range by coupling a telescope to a single-element SPAD 9 . Furthermore, room geometry reconstruction by probing a single visible wall using a picosecond laser and SPAD with TCSPC has been demonstrated 22 . Other demonstrated applications of TI include NLOS estimation of object motion and size 23 and single-viewpoint estimation of angular reflectance properties 24 .…”