Real-time 3D reconstruction from single-photon lidar data using plug-and-play point cloud denoisers

Abstract

Single-photon lidar has emerged as a prime candidate technology for depth imaging through challenging environments. Until now, a major limitation has been the significant amount of time required for the analysis of the recorded data. Here we show a new computational framework for real-time three-dimensional (3D) scene reconstruction from single-photon data. By combining statistical models with highly scalable computational tools from the computer graphics community, we demonstrate 3D reconstruction of complex outdoor scenes with processing times of the order of 20 ms, where the lidar data was acquired in broad daylight from distances up to 320 metres. The proposed method can handle an unknown number of surfaces in each pixel, allowing for target detection and imaging through cluttered scenes. This enables robust, real-time target reconstruction of complex moving scenes, paving the way for single-photon lidar at video rates for practical 3D imaging applications.

Air‐Stable Low‐Symmetry Narrow‐Bandgap 2D Sulfide Niobium for Polarization Photodetection

“…Mid‐wavelength infrared (MWIR) photodetectors are associated with significant applications such as infrared polarization imaging, [ 1 ] remote sensing, [ 2 ] free space telecommunications, [ 3 ] IR spectroscopy [ 4 ] and lidar, [ 5 ] which is of particular scientific significance and technical guidance for infrared physics technology. Traditional MWIR photodetectors including indium antimonide (InSb), [ 6 ] mercury cadmium telluride (HgCdTe) [ 7 ] and quantum‐well structures based on group III to V materials, [ 8 ] face some major challenges.…”

Full Waveform LiDAR for Adverse Weather Conditions

“…These advantages have been apparent for a number of years [40], but have suffered from long data acquisition times, too slow for most automotive applications. However, as discussed in Section VI-A1, the development of detector arrays, allied to rapid image processing techniques [41], suggest that TCSPC LiDAR systems can become more effective for future automotive requirements.…”

Spatial images from temporal data

“…In ToF approaches, the depth information is estimated by measuring the time needed by light to travel from the scene to the sensor [11]. Many recent imaging approaches, ranging from sparse-photon imaging [12][13][14] to non-line-of-sight (NLOS) imaging [15][16][17][18][19][20], also rely on computational techniques for enhancing imaging capabilities. Among the various possible computational imaging algorithms [21], machine learning (ML) [22] and, in particular, deep learning [23], provides a statistical or data-driven model for enhanced image retrieval [24].…”