A system for generating complex physically accurate sensor images for automotive applications

Liu, Zhenyi; Shen, Minghao; Zhang, Jiaqi; Liu, Shuangting; Blasinski, Henryk; Lian, Trisha; Wandell, Brian A.

doi:10.48550/arxiv.1902.04258

Cited by 3 publications

(1 citation statement)

References 4 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The authors of [30] introduce a probabilistic programming language to synthesize complex scenarios from existing domain knowledge. Another system [31] offers similar levels of control, while the camera sensor is modelled accurately at the same time. These frameworks potentially offer a way to generate improved training data for our approach.…”

Section: Introductionmentioning

confidence: 99%

Generating All the Roads to Rome: Road Layout Randomization for Improved Road Marking Segmentation

Bruls

Porav

Kunze

et al. 2019

2019 IEEE Intelligent Transportation Systems Conference (ITSC)

View full text Add to dashboard Cite

Road markings provide guidance to traffic participants and enforce safe driving behaviour, understanding their semantic meaning is therefore paramount in (automated) driving. However, producing the vast quantities of road marking labels required for training state-of-the-art deep networks is costly, time-consuming, and simply infeasible for every domain and condition. In addition, training data retrieved from virtual worlds often lack the richness and complexity of the real world and consequently cannot be used directly. In this paper, we provide an alternative approach in which new road marking training pairs are automatically generated. To this end, we apply principles of domain randomization to the road layout and synthesize new images from altered semantic labels. We demonstrate that training on these synthetic pairs improves mIoU of the segmentation of rare road marking classes during real-world deployment in complex urban environments by more than 12 percentage points, while performance for other classes is retained. This framework can easily be scaled to all domains and conditions to generate large-scale road marking datasets, while avoiding manual labelling effort.

show abstract

Section: Introductionmentioning

confidence: 99%

Generating All the Roads to Rome: Road Layout Randomization for Improved Road Marking Segmentation

Bruls

Porav

Kunze

et al. 2019

2019 IEEE Intelligent Transportation Systems Conference (ITSC)

View full text Add to dashboard Cite

show abstract

Soft Prototyping Camera Designs for Car Detection Based on a Convolutional Neural Network

Liu¹,

Lian²,

Farrell³

et al. 2019

Preprint

View full text Add to dashboard Cite

ISETAuto: Detecting vehicles with depth and radiance information

Liu

Farrell

Wandell

2021

Preprint

View full text Add to dashboard Cite

Autonomous driving applications use two types of sensor systems to identify vehiclesdepth sensing LiDAR and radiance sensing cameras. We compare the performance (average precision) of a ResNet for vehicle detection in complex, daytime, driving scenes when the input is a depth map (D = d(x,y)), a radiance image (L = r(x,y)), or both [D,L]. (1) When the spatial sampling resolution of the depth map and radiance image are equal to typical camera resolutions, a ResNet detects vehicles at higher average precision from depth than radiance. (2) As the spatial sampling of the depth map declines to the range of current LiDAR devices, the ResNet average precision is higher for radiance than depth. (3) For a hybrid system that combines a depth map and radiance image, the average precision is higher than using depth or radiance alone. We established these observations in simulation and then confirmed them using realworld data. The advantage of combining depth and radiance can be explained by noting that the two type of information have complementary weaknesses. The radiance data are limited by dynamic range and motion blur. The LiDAR data have relatively low spatial resolution. The ResNet combines the two data sources effectively to improve overall vehicle detection.

show abstract

A system for generating complex physically accurate sensor images for automotive applications

Cited by 3 publications

References 4 publications

Generating All the Roads to Rome: Road Layout Randomization for Improved Road Marking Segmentation

Generating All the Roads to Rome: Road Layout Randomization for Improved Road Marking Segmentation

Soft Prototyping Camera Designs for Car Detection Based on a Convolutional Neural Network

ISETAuto: Detecting vehicles with depth and radiance information

Contact Info

Product

Resources

About