Using Unknown Occluders to Recover Hidden Scenes

Yedidia, Adam; Baradad, Manel; Thrampoulidis, Christos; Freeman, William T.; Wornell, Gregory W.

doi:10.1109/cvpr.2019.01251

Cited by 46 publications

(22 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…There are also several contributions showing that it is possible to do NLOS imaging without picosecond scale time resolution or with non-optical signals: Inexpensive nanosecond time of flight sensors can be used to recover the hidden scene 33 , tracking can be performed using intensity based NLOS imaging 34 , occlusions are harnessed to recover images around a corner using regular cameras [35][36][37] , even describing the occlusion-aided method as a blind deconvolution problem without knowledge of the occluder 38 . Other approaches decode the hidden object from regular camera images by using a deep neural network trained with simulated data only 39 , or use acoustic 40 or long-wave infrared 41 signals to image around the corner.…”

Section: Discussionmentioning

confidence: 99%

Phasor field diffraction based reconstruction for fast non-line-of-sight imaging systems

2020

View full text Add to dashboard Cite

Non-line-of-sight (NLOS) imaging recovers objects using diffusely reflected indirect light using transient illumination devices in combination with a computational inverse method. While capture systems capable of collecting light from the entire NLOS relay surface can be much more light efficient than single pixel point scanning detection, current reconstruction algorithms for such systems have computational and memory requirements that prevent real-time NLOS imaging. Existing real-time demonstrations also use retroreflective targets and reconstruct at resolutions far below the hardware limits. Our method presented here enables the reconstruction of room-sized scenes from non-confocal, parallel multi-pixel measurements in seconds with less memory usage. We anticipate that our method will enable real-time NLOS imaging when used with emerging single-photon avalanche diode array detectors with resolution only limited by the temporal resolution of the sensor.

show abstract

Section: Discussionmentioning

confidence: 99%

Phasor field diffraction based reconstruction for fast non-line-of-sight imaging systems

2020

View full text Add to dashboard Cite

show abstract

“…In a subsequent work, the hidden scene and complex occluder structure were recovered from a video sequence. 26 Our work, which first appeared earlier this year, 24 recovers the position of the occluder (assumed to have a known shape) and a 2D color photograph of the hidden scene from a single photograph of a visible wall. In this paper, we give an overview of that approach for NLOS imaging along with new results that relate the computational field of view (CFOV) concept to the Cramér-Rao bound (CRB), new simulations, and new experimental results.…”

Section: Introductionmentioning

confidence: 97%

“…In general, this has been achieved using measurements of properties of the light scattered onto visible surfaces from the hidden scene. For example, these methods have relied on measurements of time-of-flight, [1][2][3][4][5][6][7][8][9][10][11][12][13][14][15][16][17] coherence, [18][19][20] or even intensity-only [21][22][23][24][25][26] information. NLOS imaging using acoustic 27 and long-wave infrared 28 waves have also been recent lines of work.…”

Section: Introductionmentioning

confidence: 99%

“…These methods benefit from occluding structures separating the light paths reaching a visible surface from the hidden scene. 22,23,26,[33][34][35] Vertical edges were first exploited as occluders to reconstruct a video of 1D projections of the moving parts of the hidden scene, from a video of the floor on the visible side. 22 This approach was extended by Seidel et al 35 for stationary scenes and arbitrary floor patterns, using only a single photograph.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Occlusion-based computational periscopy with consumer cameras

Murray-Bruce

Saunders

Goyal

2019

Wavelets and Sparsity XVIII

View full text Add to dashboard Cite

The ability to form images of scenes hidden from direct view would be advantageous in many applications-from improved motion planning and collision avoidance in autonomous navigation to enhanced danger anticipation for first-responders in search-and-rescue missions. Recent techniques for imaging around corners have mostly relied on time-of-flight measurements of light propagation, necessitating the use of expensive, specialized optical systems. In this work, we demonstrate how to form images of hidden scenes from intensity-only measurements of the light reaching a visible surface from the hidden scene. Our approach exploits the penumbra cast by an opaque occluding object onto a visible surface. Specifically, we present a physical model that relates the measured photograph to the radiosity of the hidden scene and the visibility function due to the opaque occluder. For a given scene-occluder setup, we characterize the parts of the hidden region for which the physical model is well-conditioned for inversion-i.e., the computational field of view (CFOV) of the imaging system. This concept of CFOV is further verified through the Cramér-Rao bound of the hidden-scene estimation problem. Finally, we present a two-step computational method for recovering the occluder and the scene behind it. We demonstrate the effectiveness of the proposed method using both synthetic and experimentally measured data.

show abstract

“…A few recent works have demonstrated NLOS images with extraordinarily high resolution without pulsed illumination 19,20 ; however, these methods are fundamentally limited to small-scale scenes and thus not practical for applications such as navigation. More prominently, a number of methods that do not capture temporal information have taken advantage of occlusions that block certain light paths [21][22][23][24][25][26][27][28][29][30][31][32] , similar to coded-aperture imaging or reference structure tomography 33 . For instance, the development of the corner camera 22 established that the high directional uncertainty created by diffuse reflection from the relay surface can be reduced by an opaque object between the relay surface and the hidden scene.…”

mentioning

confidence: 99%

Seeing around corners with edge-resolved transient imaging

et al. 2020

View full text Add to dashboard Cite

Non-line-of-sight (NLOS) imaging is a rapidly growing field seeking to form images of objects outside the field of view, with potential applications in autonomous navigation, reconnaissance, and even medical imaging. The critical challenge of NLOS imaging is that diffuse reflections scatter light in all directions, resulting in weak signals and a loss of directional information. To address this problem, we propose a method for seeing around corners that derives angular resolution from vertical edges and longitudinal resolution from the temporal response to a pulsed light source. We introduce an acquisition strategy, scene response model, and reconstruction algorithm that enable the formation of 2.5-dimensional representations—a plan view plus heights—and a 180∘ field of view for large-scale scenes. Our experiments demonstrate accurate reconstructions of hidden rooms up to 3 meters in each dimension despite a small scan aperture (1.5-centimeter radius) and only 45 measurement locations.

show abstract

Using Unknown Occluders to Recover Hidden Scenes

Cited by 46 publications

References 29 publications

Phasor field diffraction based reconstruction for fast non-line-of-sight imaging systems

Phasor field diffraction based reconstruction for fast non-line-of-sight imaging systems

Occlusion-based computational periscopy with consumer cameras

Seeing around corners with edge-resolved transient imaging

Contact Info

Product

Resources

About