Deep Learning with Attention Mechanisms for Road Weather Detection

Abstract

There is great interest in automatically detecting road weather and understanding its impacts on the overall safety of the transport network. This can, for example, support road condition-based maintenance or even serve as detection systems that assist safe driving during adverse climate conditions. In computer vision, previous work has demonstrated the effectiveness of deep learning in predicting weather conditions from outdoor images. However, training deep learning models to accurately predict weather conditions using real-world road-facing images is difficult due to: (1) the simultaneous occurrence of multiple weather conditions; (2) imbalanced occurrence of weather conditions throughout the year; and (3) road idiosyncrasies, such as road layouts, illumination, and road objects, etc. In this paper, we explore the use of a focal loss function to force the learning process to focus on weather instances that are hard to learn with the objective of helping address data imbalances. In addition, we explore the attention mechanism for pixel-based dynamic weight adjustment to handle road idiosyncrasies using state-of-the-art vision transformer models. Experiments with a novel multi-label road weather dataset show that focal loss significantly increases the accuracy of computer vision approaches for imbalanced weather conditions. Furthermore, vision transformers outperform current state-of-the-art convolutional neural networks in predicting weather conditions with a validation accuracy of 92% and an F1-score of 81.22%, which is impressive considering the imbalanced nature of the dataset.

A Review of Deep Learning Advancements in Road Analysis for Autonomous Driving

“…An innovative approach is presented in [76], where the authors introduce the usage of Vision Transformers (ViTs) models [77]. Unlike traditional CNNs, ViTs apply attention mechanisms to dynamically assign weights to pixels, focusing the model on relevant image features.…”

3-GWD : A Textural and AI-Based Approach for Real-Time Detection of Weather Disturbances Applied to Autonomous Vehicle

“…[24] for example, uses already existing architecture, like SqueezeNet, ResNet-50 and EfficientNet CNN layers to extract features, then a single fully connected layer with Softmax activation function to classify the data. [25] uses CNNs layers from well-known architectures and transformers to achieve multi-label weather classification. The work presented in [26] is another case of CNN-based weather detection algorithm using the Resnet18 framework.…”