Lightweight Object Detection Ensemble Framework for Autonomous Vehicles in Challenging Weather Conditions

Abstract

The computer vision systems driving autonomous vehicles are judged by their ability to detect objects and obstacles in the vicinity of the vehicle in diverse environments. Enhancing this ability of a self-driving car to distinguish between the elements of its environment under adverse conditions is an important challenge in computer vision. For example, poor weather conditions like fog and rain lead to image corruption which can cause a drastic drop in object detection (OD) performance. The primary navigation of autonomous vehicles depends on the effectiveness of the image processing techniques applied to the data collected from various visual sensors. Therefore, it is essential to develop the capability to detect objects like vehicles and pedestrians under challenging conditions such as like unpleasant weather. Ensembling multiple baseline deep learning models under different voting strategies for object detection and utilizing data augmentation to boost the models’ performance is proposed to solve this problem. The data augmentation technique is particularly useful and works with limited training data for OD applications. Furthermore, using the baseline models significantly speeds up the OD process as compared to the custom models due to transfer learning. Therefore, the ensembling approach can be highly effective in resource-constrained devices deployed for autonomous vehicles in uncertain weather conditions. The applied techniques demonstrated an increase in accuracy over the baseline models and were able to identify objects from the images captured in the adverse foggy and rainy weather conditions. The applied techniques demonstrated an increase in accuracy over the baseline models and reached 32.75% mean average precision (mAP) and 52.56% average precision (AP) in detecting cars in the adverse fog and rain weather conditions present in the dataset. The effectiveness of multiple voting strategies for bounding box predictions on the dataset is also demonstrated. These strategies help increase the explainability of object detection in autonomous systems and improve the performance of the ensemble techniques over the baseline models.

Autonomous Vehicle Detection and Tracking Based on Improved Yolov5 and Gmmpf in Harsh Weather Conditions

“…Rahee Walambe et al [24] presented an ensemble framework for autonomous vehicle detection in CWC. After data collection, data preprocessing was done by performing colour restoration, rotation, blurring and flipping to boost the detection performance of the ensemble approach.…”

In Rain or Shine: Understanding and Overcoming Dataset Bias for Improving Robustness Against Weather Corruptions for Autonomous Vehicles

“…Novel planning methods [8], networks for identifying objects from unknown classes [25] and multi-modal fusion transformer methods [26] have made progress in identifying anomalous features and show robust performance under diverse conditions. Specific challenges like multiscale OD [23], pedestrian detection in crowds [19] and detection under adverse weather [24] have also been tackled using ensemble methods and data augmentation.…”

RestoreX-AI: A Contrastive Approach towards Guiding Image Restoration via Explainable AI Systems

“…Taking 2D detectors forward, 3D detection expanded with Stereo RCNN [15], AVOD [13], MVLidar-Net [7] and MVF algorithm [37] bringing new perspectives to the task of object detection. Specific tasks like multiscale object detection [31], pedestrian detection in crowds [17] and detection under adverse weather [32] have also been solved using ensemble methods and data augmentation.…”