Ellipsoidal path connections for time-gated rendering

Abstract

During the last decade, we have been witnessing the continued development of new time-of-flight imaging devices, and their increased use in numerous and varied applications. However, physics-based rendering techniques that can accurately simulate these devices are still lacking: while existing algorithms are adequate for certain tasks, such as simulating transient cameras, they are very inefficient for simulating time-gated cameras because of the large number of wasted path samples. We take steps towards addressing these deficiencies, by introducing a procedure for efficiently sampling paths with a predetermined length, and incorporating it within rendering frameworks tailored towards simulating time-gated imaging. We use our open-source implementation of the above to empirically demonstrate improved rendering performance in a variety of applications, including simulating proximity sensors, imaging through occlusions, depth-selective cameras, transient imaging in dynamic scenes, and non-line-of-sight imaging. CCS Concepts: • Computing methodologies → Ray tracing.

iToF2dToF: A Robust and Flexible Representation for Data-Driven Time-of-Flight Imaging

“…Transient rendering [41]- [43] has enabled the simulation of ToF datasets, which has been essential for data-driven iToF due to the unavailability of real-world datasets with ground truth. Broadly, previous data-driven iToF models have used two types of "Input2Output" representations: Depth2Depth [6], [8], [9], [44] and iToF2Depth [3], [7].…”

“…iToF Simulator: We render transient images using Mitsub-aToF [43], [48]. The iToF simulator takes as input the transient images, sensor parameters, and frequency, to produce 4-phase sinusoid measurements.…”

“…Transient Data: For transient rendering we use MitsubaToF [43], an extension to the widely used Mitsuba renderer [48]. Previous works have used a modified pbrt-v3 [58] for time-resolved rendering [3], [9], proprietary software [8], or recent transient rendering software [6], [7], [41].…”

Temporally sliced photon primitives for time‐of‐flight rendering

“…Path tracing [Kaj86; LW93; LW96; VG95a] has the benefit of being unbiased, but traditional strategies struggle to sample valid light paths within the temporal manifold. Pediredla et al [PVG19] developed specially crafted “ellipsoidal connections” which give bidirectional path tracing some control in sampling paths with a desired temporal duration. This algorithm currently only handles surface interactions, though one of our estimators (Sec.…”

Differentiable transient rendering

“…Transient imaging, on the other hand, has already enabled many non-trivial applications in computer vision and scene recognition [Jarabo et al 2017], such as visualizing light in motion [Heide et al 2013;Velten et al 2013], looking around corners [Liu et al 2019a;O'Toole et al 2018], looking through turbid media Wu et al 2018], or the decomposition of global illumination components [Marco et al 2021;Wu et al 2014], to name a few. Transient rendering techniques, which lift the assumption that light speed is infinite and take explicitly into account light's time of flight, have thus become an increasingly relevant tool to accurately simulate scenarios and hypotheses without requiring cumbersome data capture sessions, as well as to help provide rigorous mathematical analyses [Jarabo et al 2014;Pediredla et al 2019].…”