Influence of AVC and HEVC compression on detection of vehicles through Faster R-CNN

Abstract

<div><div><div><p>Situational awareness based on the data collected by the vehicle perception sensors (i.e. LiDAR, RADAR, camera and ultrasonic sensors) is key for achieving assisted and automated driving functions in a safe and reliable way. However, the data rates generated by the sensor suite are difficult to support over traditional wired data communication networks on the vehicle, hence there is an interest in techniques that reduce the amount of sensor data to be transmitted without losing key information or introducing unacceptable delays. These techniques must be analysed in combination with the consumer of the data, which will most likely be a machine learning algorithm based on deep neural networks (DNNs). In this paper we demonstrate that by compression tuning the DNNs (i.e. transfer learning by re-training with compressed data) the DNN average precision and recall can significantly improve when uncompressed and compressed data are transmitted. This improvement is achieved independently from the compression standard used for compression-training (we used AVC and HEVC), and also when training and transmitted data use the same compression standard or different compression standards. Furthermore, the performance of the DNNs is stable when transmitting data with increasing lossy compression rate, up to a compression ratio of approximately 1200:1; above this value the performance starts to degrade. This work paves the way for the use of compressed sensor data in assisted and automated driving in combination with the optimisation of compression- tuned DNNs.</p></div></div></div>

Optimising Faster R-CNN Training to Enable Video Camera Compression for Assisted and Automated Driving Systems

“…It is worth noting that we used compressed data for two different aims: one aim is to mimic the compressed data to be transmitted on the wired vehicle communication networks (transmitted dataset, QP I ); the other aim is that the compressed data have been used also to re-train the selected NN (training dataset, QP T ). In this work, we tested our experimental flow using different combinations of training and transmitted datasets; more details can be found in [18].…”

Testing ground-truth errors in an automotive dataset for a DNN-based object detector

“…In particular, as a part of this work we have investigated a subpart of the KITTI dataset, which is widely used for bench-marking machine learning and DNNs for assisted and automated driving tasks. We have selected the KITTI MoSeg part of the full dataset, since it has temporally correlated frames and can be useful when investigating video data compression [21], [22]. The labels of the MoSeg dataset have been generated automatically using a simultaneous motion and vehicle detection DNN architecture.…”

“…After some preliminary activities based on DNNs and the KITTI MoSeg dataset [21], [22], a thorough inspection of the 349 frames in the test dataset was carried out. The test dataset was visualised by overlaying the ground truth bounding boxes onto their respective images.…”