Robust 3D Reconstruction of Dynamic Scenes From Single-Photon Lidar Using Beta-Divergences

Abstract

In this paper, we present a new algorithm for fast, online 3D reconstruction of dynamic scenes using times of arrival of photons recorded by single-photon detector arrays. One of the main challenges in 3D imaging using single-photon lidar in practical applications is the presence of strong ambient illumination which corrupts the data and can jeopardize the detection of peaks/surface in the signals. This background noise not only complicates the observation model classically used for 3D reconstruction but also the estimation procedure which requires iterative methods. In this work, we consider a new similarity measure for robust depth estimation, which allows us to use a simple observation model and a non-iterative estimation procedure while being robust to mis-specification of the background illumination model. This choice leads to a computationally attractive depth estimation procedure without significant degradation of the reconstruction performance. This new depth estimation procedure is coupled with a spatio-temporal model to capture the natural correlation between neighboring pixels and successive frames for dynamic scene analysis. The resulting online inference process is scalable and well suited for parallel implementation. The benefits of the proposed method are demonstrated through a series of experiments conducted with simulated and real single-photon lidar videos, allowing the analysis of dynamic scenes at 325 m observed under extreme ambient illumination conditions.

A Novel Pseudo-Bayesian Approach for Robust Multi-Ridge Detection and Mode Retrieval

“…In this work, we propose a novel approach for extracting the ridge associated with the IF of a signal from its time-frequency representation (TFR). Similarly to [13], we consider a simple observation model and we introduce a robust approach in order to circumvent the limitations occurring through its inaccuracy. More precisely, a variety of robust divergence [14]- [16] is used instead of the classical Kullback Leibler divergence (KLD) to account for model mismatch in the presence of noise or of several frequency components.…”

A Large-Scale Outdoor Multi-modal Dataset and Benchmark for Novel View Synthesis and Implicit Scene Reconstruction

“…Calibration. Large-scale scene calibration based on highly dynamic images has always been a difficult problem, especially for outdoor scenes with no obvious local detail differences [4,20,12]. As a common solution, we use COLMAP [31,32] to achieve multi-view 3D reconstruction, image calibration and depth image rendering.…”

Robust and Guided Bayesian Reconstruction of Single-Photon 3D Lidar Data: Application to Multispectral and Underwater Imaging

“…As indicated in (14), correlation between depth and reflectivity images is introduced through the use of W to define V . This will promote close points in space having similar depths to share similar reflectivities, in addition to exploit the multiscale depth guidance information to reject or mitigate the effect of measured outliers in both D and R. Note that reflectivity texture will be preserved by considering the R dependent exponential term in (14). The reflectivity variables r ( ) k , ∀k, , follow a gamma distribution and hence show data dependent noise levels.…”

“…Such challenges include the photon sparse regime [4]- [6] often observed for long-range imaging [7]- [9] or rapid imaging based on short acquisition times [10], [11] or adaptive imaging [12], [13]. Lidar is also sensitive to the observation environment when imaging in bright daylight conditions [14], and through obscurants or turbid media, such as underwater [11], [15], or through fog, rain [1], [16]. The latter causes photon scattering which results in the immersion of the useful signal within a high and possibly nonuniform background level [17]- [19].…”