Gated2Gated: Self-Supervised Depth Estimation from Gated Images

Abstract

We propose Gated Stereo, a high-resolution and longrange depth estimation technique that operates on active gated stereo images. Using active and high dynamic range passive captures, Gated Stereo exploits multi-view cues alongside time-of-flight intensity cues from active gating. To this end, we propose a depth estimation method with a monocular and stereo depth prediction branch which are combined in a final fusion stage. Each block is supervised through a combination of supervised and gated selfsupervision losses. To facilitate training and validation, we acquire a long-range synchronized gated stereo dataset for automotive scenarios. We find that the method achieves an improvement of more than 50 % MAE compared to the next best RGB stereo method, and 74 % MAE to existing monocular gated methods for distances up to 160 m. Our code, models and datasets are available here 1 .

Improving the Perception of Objects under Foggy Conditions in the Surrounding Environment

“…The paper utilizes three images captured by the gated camera and employs a model trained on depth images with ground-truth depth maps to accurately estimate the depth of each pixel. This procedure demonstrates remarkable performance in challenging environments, such as foggy conditions, where conventional depth sensors often falter (Walia et al 2022). Another benchmark dataset of 100 images, captured in real fog conditions, was developed by (Bijelic et al 2019).…”

Range-Intensity-Profile-Guided Gated Light Ranging and Imaging Based on a Convolutional Neural Network

“…Different from the TOF camera technique, 3D range-gated imaging utilizes gated cameras with high-spatial resolution, providing more detailed depth maps and high uniformity texture images. Therefore, the 3D range-gated imaging can obtain depth maps with spatial resolution comparable to passive imaging methods at a high frame rate and with a high depth accuracy comparable to LiDAR and TOF cameras [4,[20][21][22]. Meanwhile, due to the high gate-shutter speed of gated cameras and the advanced time controlling unit (TCU), range-gated imaging can block out irrelevant background and scattered light and obtain higher quality intensity images than the traditional active depth-sensing methods.…”

Vision-guided three-dimensional range-gated imaging based on epistemic uncertainty estimation

“…To break through traditional methods and hardware limitation constraints, Gruber et al proposed the convolutional neural network method to solve the 3DRGI problem from the visual level, 9 which can realize the 3DRGI with any pulse shape and time domain parameters. In addition, Walia et al also proposed a self-supervised 3DRGI method 10 with no need for ground truth. Vision-guided 3DRGI methods break through the hardware limitation and greatly enhance the flexibility of 3DRGI.…”

“…In addition, Walia et al. also proposed a self-supervised 3DRGI method 10 with no need for ground truth. Vision-guided 3DRGI methods break through the hardware limitation and greatly enhance the flexibility of 3DRGI.…”