Dynamic Time-of-Flight

Schober, Michael F.; Adam, A.; Yair, Omer; Mazor, Shai; Nowozin, Sebastian

doi:10.1109/cvpr.2017.26

Cited by 11 publications

(7 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These approaches require perfect knowledge of the integration and pulse profiles, which is impractical due to drift, and they provide low precision for broad gating windows in real-time capture settings. Adam et al [2], and Schober et al [50], present Bayesian methods for pulsed time-offlight imaging of room-sized scenes. These methods solve probabilistic per-pixel estimation problems using priors on depth, reflectivity and ambient light, which is possible when using nanosecond exposure profiles [2,50] for room-sized scenes.…”

Section: Related Workmentioning

confidence: 99%

“…Adam et al [2], and Schober et al [50], present Bayesian methods for pulsed time-offlight imaging of room-sized scenes. These methods solve probabilistic per-pixel estimation problems using priors on depth, reflectivity and ambient light, which is possible when using nanosecond exposure profiles [2,50] for room-sized scenes. In this work, we demonstrate that exploiting spatiotemporal scene semantics allows to recover dense and lidaraccurate depth from only three slices, with exposures two orders of magnitude longer (> 100 ns), acquired in realtime.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Gated2Depth: Real-Time Dense Lidar From Gated Images

Gruber

Julca-Aguilar²,

Bijelic

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

View full text Add to dashboard Cite

We present an imaging framework which converts three images from a gated camera into high-resolution depth maps with depth accuracy comparable to pulsed lidar measurements. Existing scanning lidar systems achieve low spatial resolution at large ranges due to mechanicallylimited angular sampling rates, restricting scene understanding tasks to close-range clusters with dense sampling. Moreover, today's pulsed lidar scanners suffer from high cost, power consumption, large form-factors, and they fail in the presence of strong backscatter. We depart from point scanning and demonstrate that it is possible to turn a low-cost CMOS gated imager into a dense depth camera with at least 80 m range -by learning depth from three gated images. The proposed architecture exploits semantic context across gated slices, and is trained on a synthetic discriminator loss without the need of dense depth labels. The proposed replacement for scanning lidar systems is real-time, handles back-scatter and provides dense depth at long ranges. We validate our approach in simulation and on real-world data acquired over 4,000 km driving in northern Europe. Data and code are available at

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Gated2Depth: Real-Time Dense Lidar From Gated Images

Gruber

Julca-Aguilar²,

Bijelic

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

View full text Add to dashboard Cite

show abstract

“…The proposed gated imager captures illumination distributed in three wide gates (> 30 m) for all sensor pixels. Gated imaging [25,6,3,62,49,2,21] allows us to capture several dense high-resolution images distributed continuously across the distances in their respective temporal bin. Additionally, back-scatter can be removed by the the distribution of early gates.…”

Section: Introductionmentioning

confidence: 99%

Gated3D: Monocular 3D Object Detection From Temporal Illumination Cues

Julca-Aguilar¹,

Taylor²,

Bijelic³

et al. 2021

Preprint

View full text Add to dashboard Cite

Today's state-of-the-art methods for 3D object detection are based on lidar, stereo, or monocular cameras. Lidar-based methods achieve the best accuracy, but have a large footprint, high cost, and mechanically-limited angular sampling rates, resulting in low spatial resolution at long ranges. Recent approaches based on low-cost monocular or stereo cameras promise to overcome these limitations but struggle in low-light or low-contrast regions as they rely on passive CMOS sensors. In this work, we propose a novel 3D object detection modality that exploits temporal illumination cues from a low-cost monocular gated imager. We propose a novel deep detector architecture, Gated3D, that is tailored to temporal illumination cues from three gated images. Gated images allow us to exploit mature 2D object feature extractors that guide the 3D predictions through a frustum segment estimation. We assess the proposed method on a novel 3D detection dataset that includes gated imagery captured in over 10,000 km of driving data. We validate that our method outperforms state-of-the-art monocular and stereo approaches at long distances. We will release our code and dataset, opening up a new sensor modality as an avenue to replace lidar in autonomous driving.

show abstract

“…While the field of automatic speech recognition is dominated by Hidden-Markov-Models (HMMs), they remain rather unpopular in computer vision related tasks. For instance CVPR, by many regarded as the top conference of computer vision, had only three out of a total of over 700 submissions in the year 2017 that were using HMMs (Koller et al 2017;Richard et al 2017;Schober et al 2017). This may be related to the comparatively poor image modelling capabilities of Gaussian Mixture Models (GMMs), which had been traditionally used to model the observation probabilities within such a framework.…”

Section: Introductionmentioning

confidence: 99%

Deep Sign: Enabling Robust Statistical Continuous Sign Language Recognition via Hybrid CNN-HMMs

et al. 2018

View full text Add to dashboard Cite

This manuscript introduces the end-to-end embedding of a CNN into a HMM, while interpreting the outputs of the CNN in a Bayesian framework. The hybrid CNN-HMM combines the strong discriminative abilities of CNNs with the sequence modelling capabilities of HMMs. Most current approaches in the field of gesture and sign language recognition disregard the necessity of dealing with sequence data both for training and evaluation. With our presented end-to-end embedding we are able to improve over the state-of-the-art on three challenging benchmark continuous sign language recognition tasks by between 15 and 38% relative reduction in word error rate and up to 20% absolute. We analyse the effect of the CNN structure, network pretraining and number of hidden states. We compare the hybrid modelling to a tandem approach and evaluate the gain of model combination.

show abstract

Dynamic Time-of-Flight

Cited by 11 publications

References 29 publications

Gated2Depth: Real-Time Dense Lidar From Gated Images

Gated2Depth: Real-Time Dense Lidar From Gated Images

Gated3D: Monocular 3D Object Detection From Temporal Illumination Cues

Deep Sign: Enabling Robust Statistical Continuous Sign Language Recognition via Hybrid CNN-HMMs

Contact Info

Product

Resources

About