Error Backprojection Algorithms for Non-Line-of-Sight Imaging

Manna, Marco; Kine, Fiona; Breitbach, Eric; Jackson, James H.; Sultan, Talha; Velten, Andreas

doi:10.1109/tpami.2018.2843363

Cited by 55 publications

(23 citation statements)

References 34 publications

(44 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Fur-thermore the complete 3D retrieval of the hidden scene often requires prohibitive time resources with the 3D imaging of a moving object, although progress is being made to reduce the acquisition and reconstruction times [6,19], with the goal of reaching second or even sub-second times. Additionally, the spatial resolution of the retrieval can be improved by using iterative backprojection algorithms that however, typically require longer computational times [20,21]. Another rapidly evolving imaging technique is based on so-called single-pixel cameras [22].…”

mentioning

confidence: 99%

Non-Line-of-Sight Three-Dimensional Imaging with a Single-Pixel Camera

et al. 2019

View full text Add to dashboard Cite

Real time, high resolution 3D reconstruction of scenes hidden from the direct field of view is a challenging field of research with applications in real-life situations related e.g. to surveillance, self-driving cars and rescue missions. Most current techniques recover the 3D structure of a non-lineof-sight (NLOS) static scene by detecting the return signal from the hidden object on a scattering observation area. Here, we demonstrate the full colour retrieval of the 3D shape of a hidden scene by coupling back-projection imaging algorithms with the high-resolution time-of-flight information provided by a single-pixel camera. By using a high e ciency Single-Photon Avalanche Diode (SPAD) detector, this technique provides the advantage of imaging with no mechanical scanning parts, with acquisition times down to sub-seconds.

show abstract

mentioning

confidence: 99%

Non-Line-of-Sight Three-Dimensional Imaging with a Single-Pixel Camera

et al. 2019

View full text Add to dashboard Cite

show abstract

“…Such a dataset is not suitable for implementing the virtual point light source described in Eq. (5). Instead, we can model an illumination wavefront that is focused on x v :…”

Section: Timementioning

confidence: 99%

“…After theoretical exploration of the problem 1,2 , the first experimental demonstration of NLOS imaging used a filtered backprojection (FBP) algorithm 3,4 similar to inverse methods used in computed tomography. Modified FBP algorithms such as error backprojection 5 and Laplacian of Gaussian (LOG) FBP 6 can provide high quality reconstructions, but have a high computational complexity and take minutes to hours to execute on a desktop computer. Buttafava et al 7 show that it is possible to use a gated Single-Photon Avalanche Diode (SPAD) for NLOS imaging.…”

mentioning

confidence: 99%

Phasor field diffraction based reconstruction for fast non-line-of-sight imaging systems

2020

Self Cite

View full text Add to dashboard Cite

Non-line-of-sight (NLOS) imaging recovers objects using diffusely reflected indirect light using transient illumination devices in combination with a computational inverse method. While capture systems capable of collecting light from the entire NLOS relay surface can be much more light efficient than single pixel point scanning detection, current reconstruction algorithms for such systems have computational and memory requirements that prevent real-time NLOS imaging. Existing real-time demonstrations also use retroreflective targets and reconstruct at resolutions far below the hardware limits. Our method presented here enables the reconstruction of room-sized scenes from non-confocal, parallel multi-pixel measurements in seconds with less memory usage. We anticipate that our method will enable real-time NLOS imaging when used with emerging single-photon avalanche diode array detectors with resolution only limited by the temporal resolution of the sensor.

show abstract

“…More importantly, unlike linear backprojection used in standard emission or absorption tomography, ellipsoidal backprojection is not the adjoint of a physically motivated forward light transport operator. This necessitates heavy heuristic filtering [Manna et al 2018], and the reconstructed shapes are typically flat and low in detail. On the other hand, algorithms based on ellipsoidal backprojection generally have much shorter runtimes than our approach, since they don't require a global optimization scheme.…”

Section: Motivationmentioning

confidence: 99%

Non-line-of-sight Reconstruction Using Efficient Transient Rendering

Iseringhausen

Hullin

2020

ACM Trans. Graph.

View full text Add to dashboard Cite

Object Object(a) (b) (c) Fig. 1. (a) The challenge of looking around the corner deals with the recovery of information about objects beyond the direct line of sight. In this illustration of a setting proposed by Velten et al. [2012], an unknown object is located in front of a wall, but additional obstacles occlude the object from any optical devices like light sources or cameras. Our only source of information are therefore indirect reflections off other surfaces (here, a planar "wall"). A point on the wall that is illuminated by an ultrashort laser pulse turns into an omnidirectional source of indirect light ("laser spot"). After scattering off the unknown object, some of that light arrives back at the wall, where it forms an optical "echo" or space-time response (shown are 2D slices) that can be picked up by a suitable camera. Locations on the wall can be interpreted as omnidirectional detector pixels that receive different mixtures of backscattered light contributions at different times. We assume that neither camera nor laser can directly illuminate or observe the object, leaving us with the indirect optical space-time response as the only source of information. Note that for the sake of clarity, laser source, camera, and occluder are not shown here. The complete setup is illustrated in Figure 2. (b) We propose a novel transient renderer to simulate such indirectly scattered light transport efficiently enough for use as a forward model in inverse problems. In this artistic visualization, light contributions removed by the shadow test are marked in red, and the net intensity in blue. Together with an optimization algorithm, the renderer can be used to reconstruct the geometry of objects outside the line of sight. (c) Left to right: ground-truth object geometry; reconstruction using a state-of-the-art method (ellipsoidal backprojection); reconstruction using the technique presented in this paper. Top row: BunnyGI dataset; bottom row: Mannequin1Laser dataset. Our method relies on highly efficient and near-physical forward simulation, and it exemplifies the use of computer graphics as a technical tool to solve inverse problems in other fields.Being able to see beyond the direct line of sight is an intriguing prospective and could benefit a wide variety of important applications. Recent work has demonstrated that time-resolved measurements of indirect diffuse light contain valuable information for reconstructing shape and reflectance properties of objects located around a corner. In this paper, we introduce a novel reconstruction scheme that, by design, produces solutions that are consistent with state-of-the-art physically-based rendering. Our method combines an efficient forward model (a custom renderer for time-resolved three-bounce indirect light transport) with an optimization framework to reconstruct object geometry in an analysis-by-synthesis sense. We evaluate our algorithm on a variety of synthetic and experimental input data, and show that it gracefully handles uncooperative scenes with high levels of no...

show abstract

Error Backprojection Algorithms for Non-Line-of-Sight Imaging

Cited by 55 publications

References 34 publications

Non-Line-of-Sight Three-Dimensional Imaging with a Single-Pixel Camera

Non-Line-of-Sight Three-Dimensional Imaging with a Single-Pixel Camera

Phasor field diffraction based reconstruction for fast non-line-of-sight imaging systems

Non-line-of-sight Reconstruction Using Efficient Transient Rendering

Contact Info

Product

Resources

About