Wave-based non-line-of-sight imaging using fast f-k migration

Abstract

Imaging objects outside a camera's direct line of sight has important applications in robotic vision, remote sensing, and many other domains. Time-of-flight-based non-line-of-sight (NLOS) imaging systems have recently demonstrated impressive results, but several challenges remain. Image formation and inversion models have been slow or limited by the types of hidden surfaces that can be imaged. Moreover, non-planar sampling surfaces and non-confocal scanning methods have not been supported by efficient NLOS algorithms. With this work, we introduce a wave-based image formation model for the problem of NLOS imaging. Inspired by inverse methods used in seismology, we adapt a frequency-domain method, f-k migration, for solving the inverse NLOS problem. Unlike existing NLOS algorithms, f-k migration is both fast and memory efficient, it is robust to specular and other complex reflectance properties, and we show how it can be used with non-confocally scanned measurements as well as for non-planar sampling surfaces. f-k migration is more robust to measurement noise than alternative methods, generally produces better quality reconstructions, and is easy to implement. We experimentally validate our algorithms with a new NLOS imaging system that records room-sized scenes outdoors under indirect sunlight, and scans persons wearing retroreflective clothing at interactive rates.

Phasor field diffraction based reconstruction for fast non-line-of-sight imaging systems

“…Among the fastest current reconstruction methods, O'Toole et al 8 propose a Light Cone Transform (LCT) method based on co-located illumination and detection points and acquire all measurements through a scanning process of the relay wall (socalled confocal acquisition setup). Lindell et al 9 demonstrate another reconstruction method for confocal data transferred from seismic imaging which is called FK Migration. Both algorithms rely on 3D convolutions allowing for fast reconstruction and demonstrate the ability to recover complex scenes from confocal measurements 9 .…”

Non-Line-of-Sight Imaging

“…Over the last few years, a variety of different approaches to addressing the NLOS problem have been proposed. Some of these focus on advanced measurement systems, using femtosecond and picosecond time-resolved detectors [2][3][4][5], interferometry [6,7], acoustic systems [8], passive imaging systems [9][10][11], or thermal imaging system [12,13], while others explore various models of light transport that make certain assumptions on reflectance or other properties of the hidden scenes. At the convergence of physics, signal processing, optics, and electronics, NLOS imaging is an interdisciplinary challenge that has seen much progress over the last few years.…”

“…It still seems unclear, however, what the "best" representation for general NLOS imaging is. Wave Optics Models, rather than the above outlined geometric optics model, have recently been explored for transient imaging configurations with time-resolved detectors and pulsed light sources [4,5,[86][87][88][89] (see Figs. 4,5).…”

“…Wave Optics Models, rather than the above outlined geometric optics model, have recently been explored for transient imaging configurations with time-resolved detectors and pulsed light sources [4,5,[86][87][88][89] (see Figs. 4,5). In these methods, the light transport in the hidden scene is modeled using the time-dependent wave equation or other models from physical optics.…”

“…The wave-based nature of this inverse method is unique in being robust to objects with diverse and complex reflectance properties, such as the glossy dragon, the diffuse statue, and the reflective disco ball shown in this scene. All volumes are rendered as maximum intensity projections.Figure adapted from[5].…”