Bayesian Time-of-Flight for Realtime Shape, Illumination and Albedo

Abstract

Abstract-We propose a computational model for shape, illumination and albedo inference in a pulsed time-of-flight (TOF) camera. In contrast to TOF cameras based on phase modulation, our camera enables general exposure profiles. This results in added flexibility and requires novel computational approaches. To address this challenge we propose a generative probabilistic model that accurately relates latent imaging conditions to observed camera responses. While principled, realtime inference in the model turns out to be infeasible, and we propose to employ efficient non-parametric regression trees to approximate the model outputs. As a result we are able to provide, for each pixel, at video frame rate, estimates and uncertainty for depth, effective albedo, and ambient light intensity. These results we present are state-of-the-art in depth imaging. The flexibility of our approach allows us to easily enrich our generative model. We demonstrate that by extending the original single-path model to a two-path model, capable of describing some multipath effects. The new model is seamlessly integrated in the system at no additional computational cost. Our work also addresses the important question of optimal exposure design in pulsed TOF systems. Finally, for benchmark purposes and to obtain realistic empirical priors of multipath and insights into this phenomena, we propose a physically accurate simulation of multipath phenomena.

Gated2Depth: Real-Time Dense Lidar From Gated Images

“…These approaches require perfect knowledge of the integration and pulse profiles, which is impractical due to drift, and they provide low precision for broad gating windows in real-time capture settings. Adam et al [2], and Schober et al [50], present Bayesian methods for pulsed time-offlight imaging of room-sized scenes. These methods solve probabilistic per-pixel estimation problems using priors on depth, reflectivity and ambient light, which is possible when using nanosecond exposure profiles [2,50] for room-sized scenes.…”

What Are Optimal Coding Functions for Time-of-Flight Imaging?

“…It is computationally intensive to even evaluate this metric numerically as it involves computing a double integral over highdimensional unknown and measurement spaces. This precludes the development of an e cient numerical or an exhaustive searchbased optimization procedure as well due to prohibitively high computational costs [Adam et al 2016]. We take a di erent approach.…”

“…Lasers and didoes such as those increasingly being used in ToF systems can emit short pulses with high peak power [Adam et al 2016;Kolb et al 2010;Tadmor et al 2014], but low average power due to energy consumption constraints and eye safety. Such sources can closely approximate an impulse modulation function, and thus, achieve high performance.…”

Bidirectional Rendering of Vector Light Transport

“…The emergence of transient imaging [JMMG17] have brought an increased interest on simulating transient light transport in graphics and vision [Jar12, PBSC14, ADY*17]. Jarabo et al .…”