Road scenes analysis in adverse weather conditions by polarization-encoded images and adapted deep learning

Abstract

Object detection in road scenes is necessary to develop both autonomous vehicles and driving assistance systems. Even if deep neural networks for recognition task have shown great performances using conventional images, they fail to detect objects in road scenes in complex acquisition situations. In contrast, polarization images, characterizing the light wave, can robustly describe important physical properties of the object even under poor illumination or strong reflections. This paper shows how non-conventional polarimetric imaging modality overcomes the classical methods for object detection especially in adverse weather conditions. The efficiency of the proposed method is mostly due to the high power of the polarimetry to discriminate any object by its reflective properties and on the use of deep neural networks for object detection. Our goal by this work, is to prove that polarimetry brings a real added value compared with RGB images for object detection. Experimental results on our own dataset composed of road scene images taken during adverse weather conditions show that polarimetry together with deep learning can improve the state-of-the-art by about 20% to 50% on different detection tasks.

Multimodal Polarimetric And Color Fusion For Road Scene Analysis In Adverse Weather Conditions

“…The combination of these two complementary modalities largely reduces the false alarm rate and thus improves the detection accuracy. In our previous work [11], polarimetric images are used to improve object detection in road scenes under fog. New data formats with polarimetric features are created to better characterize objects.…”

A new multimodal RGB and polarimetric image dataset for road scenes analysis

“…In a previous work [7], a first version of a polarimetric dataset containing diverse polarimetric encoded road scenes in sunny and foggy weather conditions was presented. This dataset was limited as the RGB counterpart of the polarimetric image was missing .…”

“…Their reflective system enabled to characterize the nature of the object detected and to avoid the confusion between an object and its ghost equivalent. Recently, Blin et al [7] used polarimetric images in order to improve object detection in road scenes in adverse weather conditions. New data formats with polarimetric features were constitued to best characterize objects.…”

Optimizing Deep-Neural-Network-Driven Autonomous Race Car Using Image Scaling

“…al. [11] has demonstrated road scene analysis by using polarizationencoded images. We gathered over 10,000 images and each image has its own JSON file associated with it as shown in Fig.…”