Multi-Modal Non-Line-of-Sight Passive Imaging

Abstract

We consider the non-line-of-sight (NLOS) imaging of an object using the light reflected off a diffusive wall. The wall scatters incident light such that a lens is no longer useful to form an image. Instead, we exploit the 4D spatial coherence function to reconstruct a 2D projection of the obscured object. The approach is completely passive in the sense that no control over the light illuminating the object is assumed and is compatible with the partially coherent fields ubiquitous in both the indoor and outdoor environments. We formulate a multi-criteria convex optimization problem for reconstruction, which fuses the reflected field's intensity and spatial coherence information at different scales. Our formulation leverages established optics models of light propagation and scattering and exploits the sparsity common to many images in different bases. We also develop an algorithm based on the alternating direction method of multipliers to efficiently solve the convex program proposed. A means for analyzing the null space of the measurement matrices is provided as well as a means for weighting the contribution of individual measurements to the reconstruction. This paper holds promise to advance passive imaging in the challenging NLOS regimes in which the intensity does not necessarily retain distinguishable features and provides a framework for multimodal information fusion for efficient scene reconstruction.

Recent Advances on Non-Line-of-Sight Imaging: Conventional Physical Models, Deep Learning, and New Scenes

“…Batarseh et al used the Dual-Phase Sagnac Interferometer (DuPSaI) to measure the spatial coherence of the light field [7], thereby completing the detection and positioning of the hidden broadband ambient light source. Beckus et al combined spatial coherence and intensity information to complete multi-modal passive NLOS imaging [72]. In addition to spatial coherence, temporal coherence can also be used to obtain depth information.…”

“…( 11) is larger, meaning more ill-posed. In addition to the above-mentioned intensity-based forward model, imaging models based on other principles and settings (including the Fresnel model [72], partial occlusion [6], [73], spatial coherence [7], [72], and polarizers [74]) can all be discretized to inverse optimization problems similar with Eq. ( 11).…”

“…The combination of coherence and intensity information can effectively overcome this limitation. For example, multi-modal passive NLOS imaging that combines intensity information and spatial coherence can achieve promising results [72].…”

Thermal non-line-of-sight imaging from specular and diffuse reflections

“…Saunders et al [8] proposed a passive NLOS imaging which requires only a single image captured with an ordinary camera to recover a partially hidden scene and the position of an opaque object in the NLOS scene. Beckus et al [9] also proposed a passive NLOS imaging by a multi-modal fusion of the intensity and the spatial coherence function.…”

Non-Line-of-Sight Location With Gauss Filtering Algorithm Based on a Model of Photon Flight

“…This technology has many valuable applications. Its wide application in complex situations such as rescue, autopilot, and antiterrorist operations could reduce casualties [1]. Furthermore, it could be applied to real-time path planning to assist unmanned devices such as drones, sweeping robots, and automated vehicles.…”