Instance Segmentation in CARLA: Methodology and Analysis for Pedestrian-oriented Synthetic Data Generation in Crowded Scenes

Abstract

The evaluation of camera-based perception functions in automated driving (AD) is a significant challenge and requires large-scale high-quality datasets. Recently proposed metrics for safety evaluation additionally require detailed per-instance annotations of dynamic properties such as distance and velocities that may not be available in openly accessible AD datasets. Synthetic data from 3D simulators like CARLA may provide a solution to this problem as labeled data can be produced in a structured manner. However, CARLA currently lacks instance segmentation ground truth. In this paper, we present a back projection pipeline that allows us to obtain accurate instance segmentation maps for CARLA, which is necessary for precise per-instance ground truth information. Our evaluation results show that per-pedestrian depth aggregation obtained from our instance segmentation is more precise than previously available approximations based on bounding boxes especially in the context of crowded scenes in urban automated driving.

Using a Monocular Camera for 360∘ Dynamic Object Instance Segmentation in Traffic

“…One of the main fields of application is lane and road marking detection, where instance segmentation is used to improve detecting precision [11][12][13]. Similarly to our research, other papers have applied instance segmentation on dynamic traffic elements like vehicles [14,15] and pedestrians [16,17]. For example, the authors in [18] propose an implementation of a state-of-the-art Mask R-CNN method using a transfer learning technique for vehicle detection via instance-wise segmentation, which produces a bounding box and object mask simultaneously.…”

Training and Validation of Visual Perception Functions for Autonomous Driving with Synthetic Data

“…These datasets are valuable for benchmarking cross-domain adaptation methods but do not allow for additional creation of corner-case data. Here, autonomous driving simulators such as Carla [DRC+17] and the LGSVL simulator [RST+20] prove to be beneficial [LGH+21b;GHA21]. Procedural methods for road generation [PJX+20] can enhance the capabilities of these methods.…”