Robust Deep Neural Object Detection and Segmentation for Automotive Driving Scenario with Compressed Image Data

Abstract

Deep neural object detection or segmentation networks are commonly trained with pristine, uncompressed data. However, in practical applications the input images are usually deteriorated by compression that is applied to efficiently transmit the data. Thus, we propose to add deteriorated images to the training process in order to increase the robustness of the two state-of-the-art networks Faster and Mask R-CNN. Throughout our paper, we investigate an autonomous driving scenario by evaluating the newly trained models on the Cityscapes dataset that has been compressed with the upcoming video coding standard Versatile Video Coding (VVC). When employing the models that have been trained with the proposed method, the weighted average precision of the R-CNNs can be increased by up to 3.68 percentage points for compressed input images, which corresponds to bitrate savings of nearly 48 %.

Domain Adaptation for Unknown Image Distortions in Instance Segmentation

“…In [8], Fischer et al show that robustness against distortion may be obtained either by including the degraded images into the training data, or by fine-tuning a network trained on pristine data on the respective distortion. Therefore, we choose the same pre-trained Mask R-CNN [14] provided in [15] and perform a fine-tuning on training data distorted by means of the learned pristine-to-distortion mapping function C.…”

“…In previous work, it has been shown that the performance of Deep Neural Network (DNN)-based techniques for machine vision decreases if the input images or videos are subject to such distortions. This negative impact has been shown for image classification [1][2][3][4][5], semantic segmentation [6], object detection [7,8], instance segmentation [8] and license plate recognition [9]. A common approach to encounter this decrease in performance is to enlarge data sets by data augmentation, i.e., extending them with modified versions of the original images by applying expected distortions synthetically [10].…”

Video Coding for Machines: Large-Scale Evaluation of Deep Neural Networks Robustness to Compression Artifacts for Semantic Segmentation

“…This is because some studies only compare themselves to HEVC test Model (HM) or VVC Test Model (VTM) with few Quantization Parameter (QP) after proposing a new method to reach better tradeoffs between vision task performance and bitrate [14]- [17], [19], [25], [26]. Some papers also evaluate DNN resilience to JPEG/JPEG2000 compression [10], [12], [15], [18], [21], [24], [27], [28], [30], AVC [1], [31] or auto-encoders [18], [26], [27], but no paper consider all mentioned image and video codec generations in a unified framework (II). Note that older codecs such as JPEG or AVC achieve lower trade-offs between bitrate and vision task performance, but their low-complexity compared to modern codecs makes them more suitable to some applications using low-power devices [31], especially when hardware implementation of AVC encoders is still widespread nowadays.…”

“…Few papers attempted to (I) use compressed data at training time in order to improve DNN resilience to compression artifacts [15], [20], [22], [24], [27], [28], [34]. All studies converge to show that adding compressed data at training time allows to reach much higher trade-offs between bitrate and vision task performance.…”

“…All studies converge to show that adding compressed data at training time allows to reach much higher trade-offs between bitrate and vision task performance. While training and evaluation with the same encoding configuration allows to reach the best trade-offs, training and evaluating with different codec is still beneficial for DNN resilience [15], [27], [28].…”