Content adaptive video compression for autonomous vehicle remote driving

Abstract

Autonomous vehicles are a promising solution to traffic congestion, air pollution, accidents, and wasted time and resources. However, remote driver intervention may be necessary for extreme situations to ensure safe roadside parking or complete remote takeover. In such cases, high-quality real-time video streaming is crucial for practical remote driving. In a preliminary study, we already presented a region of interest (ROI) HEVC data compression where the image was segmented into two categories of ROI and background, allocating more bandwidth to the ROI, yielding an improvement in the visibility of the classes that essential for driving while transmitting the background with lesser quality. However, migrating bandwidth to the large ROI portion of the image doesn't substantially improve the quality of traffic signs and lights. This work categorized the ROIs into either background, weak ROI, or strong ROI. The simulation-based approach uses a photo-realistic driving scenario database created with the Cognata self-driving car simulation platform. We use semantic segmentation to categorize the compression quality of a Coding Tree Unit (CTU) according to each pixel class. A background CTU can contain only sky, trees, vegetation, or building classes. Essentials for remote driving include significant classes such as roads, road marks, cars, and pedestrians. And most importantly, traffic signs and traffic lights. We apply thresholds to decide if the number of pixels in a CTU of a particular category is enough to declare it as belonging to the strong or weak ROI. Then, we allocate the bandwidth according to the CTU categories. Our results show that the perceptual quality of traffic signs, especially textual signs and traffic lights, improves significantly by up to 5.5 dB compared to the only background and foreground partition, while the weak ROI classes at least retain their original quality.

Semantic-Aware Video Compression for Automotive Cameras

“…Some content-adaptive video compression techniques for AVs have been recently proposed. However, the work from Dror et al is for remote AV control, whereas Wang et al only presents some preliminary results with semantic-aware compression and H.264 [27], [12].…”

A Two-stage H.264 based Video Compression Method for Automotive Cameras

“…Several content-aware compression techniques use video segmentation to separate the foreground objects from the background. One recent work has proposed a content-adaptive video compression for AVs remote control application [13]. It uses a simulated dataset for the ROI and non-ROI compression by varying the quantisation parameter.…”