Fast back-projection for non-line of sight reconstruction

Arellano, Víctor Manuel; Gutiérrez, Diego; Jarabo, Adrián

doi:10.1364/oe.25.011574

Cited by 94 publications

(34 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…In this paper, we focus on the archetypal challenge of reconstructing the shape of an unknown object from 3-bounce indirect and (more or less) diffuse reflections off a planar wall (Figure 1(a)) [Kirmani et al 2009]. The overwhelming majority of approaches to this class of problem rely on ellipsoidal backprojection, where intensity measurements are smeared out over the loci in space (ellipsoidal shells) that correspond to plausible scattering locations under the given geometric constraints [Arellano et al 2017;Buttafava et al 2015;Gariepy et al 2016;Kadambi et al 2016;Velten et al 2012]. Ellipsoidal backprojection implicitly assumes that the object is a volumetric scatterer, and it does not take into account surface orientation and self-occlusion of the object.…”

Section: Motivationmentioning

confidence: 99%

“…The reconstruction strategies can be roughly grouped in two classes. One major group is formed by backprojection approaches where each input measurement casts votes on those locations in the scene where the light could have been scattered [Arellano et al 2017;Buttafava et al 2015;Gariepy et al 2016;Kadambi et al 2016;Laurenzis and Velten 2014;Velten et al 2012]. A smaller but more diverse group of work relies on the use of forward models to arrive at a scene hypothesis that best agrees with the measured data.…”

Section: Analysis Of Transient Light Transport and Looking Around Cormentioning

confidence: 99%

See 1 more Smart Citation

Non-line-of-sight Reconstruction Using Efficient Transient Rendering

Iseringhausen

Hullin

2020

ACM Trans. Graph.

View full text Add to dashboard Cite

Object Object(a) (b) (c) Fig. 1. (a) The challenge of looking around the corner deals with the recovery of information about objects beyond the direct line of sight. In this illustration of a setting proposed by Velten et al. [2012], an unknown object is located in front of a wall, but additional obstacles occlude the object from any optical devices like light sources or cameras. Our only source of information are therefore indirect reflections off other surfaces (here, a planar "wall"). A point on the wall that is illuminated by an ultrashort laser pulse turns into an omnidirectional source of indirect light ("laser spot"). After scattering off the unknown object, some of that light arrives back at the wall, where it forms an optical "echo" or space-time response (shown are 2D slices) that can be picked up by a suitable camera. Locations on the wall can be interpreted as omnidirectional detector pixels that receive different mixtures of backscattered light contributions at different times. We assume that neither camera nor laser can directly illuminate or observe the object, leaving us with the indirect optical space-time response as the only source of information. Note that for the sake of clarity, laser source, camera, and occluder are not shown here. The complete setup is illustrated in Figure 2. (b) We propose a novel transient renderer to simulate such indirectly scattered light transport efficiently enough for use as a forward model in inverse problems. In this artistic visualization, light contributions removed by the shadow test are marked in red, and the net intensity in blue. Together with an optimization algorithm, the renderer can be used to reconstruct the geometry of objects outside the line of sight. (c) Left to right: ground-truth object geometry; reconstruction using a state-of-the-art method (ellipsoidal backprojection); reconstruction using the technique presented in this paper. Top row: BunnyGI dataset; bottom row: Mannequin1Laser dataset. Our method relies on highly efficient and near-physical forward simulation, and it exemplifies the use of computer graphics as a technical tool to solve inverse problems in other fields.Being able to see beyond the direct line of sight is an intriguing prospective and could benefit a wide variety of important applications. Recent work has demonstrated that time-resolved measurements of indirect diffuse light contain valuable information for reconstructing shape and reflectance properties of objects located around a corner. In this paper, we introduce a novel reconstruction scheme that, by design, produces solutions that are consistent with state-of-the-art physically-based rendering. Our method combines an efficient forward model (a custom renderer for time-resolved three-bounce indirect light transport) with an optimization framework to reconstruct object geometry in an analysis-by-synthesis sense. We evaluate our algorithm on a variety of synthetic and experimental input data, and show that it gracefully handles uncooperative scenes with high levels of no...

show abstract

Section: Motivationmentioning

confidence: 99%

Section: Analysis Of Transient Light Transport and Looking Around Cormentioning

confidence: 99%

Non-line-of-sight Reconstruction Using Efficient Transient Rendering

Iseringhausen

Hullin

2020

ACM Trans. Graph.

View full text Add to dashboard Cite

show abstract

“…Impulse Non-Line-of-Sight-Imaging A major line of research [43,54,17,42,53,3,45,40,58] proposes to acquire transient images directly, by sending pulses of light into the scene and capturing the response with detectors capable of high temporal sampling. While the streak camera setup from Velten et al [55] allows for temporal precision of < 10 ps, corresponding to a path length of 3 mm, the high instrumentation cost and sensitivity has sparked work on single photon avalanche diodes (SPADs) as a detector alternative [7,40].…”

Section: Related Workmentioning

confidence: 99%

“…To tackle the lack of angular resolution, a number of NLOS approaches have been described that temporally probe the light-transport in the scene, thereby unmixing light path contributions by their optical path length [1,30,36,43] and effectively trading angular with temporal resolution. To acquire temporally resolved images of light transport, existing methods either directly sample the temporal impulse response of the scene by recording the temporal echoes of laser pulses [54,43,17,7,53,3,42], or they use amplitude-coded illumination and time-of-flight sensors [21,26,25]. While amplitude coding approaches suffer from low temporal resolution due to sensor demodulation bandwidth limitations [32] and the corresponding ill-posed inverse problem [19], direct probing methods achieve high temporal resolution already in the acquisition phase, but in turn require ultra-short pulsed laser illumination and detectors with < 10 ps temporal resolution for macroscopic scenes.…”

Section: Introductionmentioning

confidence: 99%

Steady-State Non-Line-Of-Sight Imaging

Chen

Daneau

Brosseau³

et al. 2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

View full text Add to dashboard Cite

Conventional intensity cameras recover objects in the direct line-of-sight of the camera, whereas occluded scene parts are considered lost in this process. Non-line-of-sight imaging (NLOS) aims at recovering these occluded objects by analyzing their indirect reflections on visible scene surfaces. Existing NLOS methods temporally probe the indirect light transport to unmix light paths based on their travel time, which mandates specialized instrumentation that suffers from low photon efficiency, high cost, and mechanical scanning. We depart from temporal probing and demonstrate steady-state NLOS imaging using conventional intensity sensors and continuous illumination. Instead of assuming perfectly isotropic scattering, the proposed method exploits directionality in the hidden surface reflectance, resulting in (small) spatial variation of their indirect reflections for varying illumination. To tackle the shapedependence of these variations, we propose a trainable architecture which learns to map diffuse indirect reflections to scene reflectance using only synthetic training data. Relying on consumer color image sensors, with high fill factor, high quantum efficiency and low read-out noise, we demonstrate high-fidelity color NLOS imaging for scene configurations tackled before with picosecond time resolution.

show abstract

“…The emergence of transient imaging has led to a vast number of applications in graphics and vision [JMMG17], where the ability of sensing the world at extreme high temporal resolution allows new applications such as imaging light in motion [VWJ*13], appearance capture [NZV*11], geometry reconstruction [BH04, MHM*17] or vision through media [Bus05, WJS*18] and around the corner [VWG*12, AGJ].…”

Section: Introductionmentioning

confidence: 99%

Progressive Transient Photon Beams

Marco

Guillén

Jarosz

et al. 2019

Computer Graphics Forum

Self Cite

View full text Add to dashboard Cite

In this work, we introduce a novel algorithm for transient rendering in participating media. Our method is consistent, robust and is able to generate animations of time‐resolved light transport featuring complex caustic light paths in media. We base our method on the observation that the spatial continuity provides an increased coverage of the temporal domain, and generalize photon beams to transient‐state. We extend stead‐state photon beam radiance estimates to include the temporal domain. Then, we develop a progressive variant of our approach which provably converges to the correct solution using finite memory by averaging independent realizations of the estimates with progressively reduced kernel bandwidths. We derive the optimal convergence rates accounting for space and time kernels, and demonstrate our method against previous consistent transient rendering methods for participating media.

show abstract

Fast back-projection for non-line of sight reconstruction

Cited by 94 publications

References 18 publications

Non-line-of-sight Reconstruction Using Efficient Transient Rendering

Non-line-of-sight Reconstruction Using Efficient Transient Rendering

Steady-State Non-Line-Of-Sight Imaging

Progressive Transient Photon Beams

Contact Info

Product

Resources

About