Towards Enhancing Traffic Sign Recognition through Sliding Windows

Abstract

Automatic Traffic Sign Detection and Recognition (TSDR) provides drivers with critical information on traffic signs, and it constitutes an enabling condition for autonomous driving. Misclassifying even a single sign may constitute a severe hazard, which negatively impacts the environment, infrastructures, and human lives. Therefore, a reliable TSDR mechanism is essential to attain a safe circulation of road vehicles. Traffic Sign Recognition (TSR) techniques that use Machine Learning (ML) algorithms have been proposed, but no agreement on a preferred ML algorithm nor perfect classification capabilities were always achieved by any existing solutions. Consequently, our study employs ML-based classifiers to build a TSR system that analyzes a sliding window of frames sampled by sensors on a vehicle. Such TSR processes the most recent frame and past frames sampled by sensors through (i) Long Short-Term Memory (LSTM) networks and (ii) Stacking Meta-Learners, which allow for efficiently combining base-learning classification episodes into a unified and improved meta-level classification. Experimental results by using publicly available datasets show that Stacking Meta-Learners dramatically reduce misclassifications of signs and achieved perfect classification on all three considered datasets. This shows the potential of our novel approach based on sliding windows to be used as an efficient solution for TSR.

Robust Traffic Sign Recognition Against Camera Failures

“…The majority of works in the literature processes a frame using a unique classifier, except few studies [17], [54], [55] which process sequences of frames.…”

“…In addition, we build classifiers that can process multiple frames in a sliding window [17]. The structure of this classifier is described in Fig.…”

“…Automatic TSR systems embed Machine Learning (ML), and especially Deep Neural Network (DNN) classifiers which process image frames captured by cameras installed on the vehicle. Those TSR systems are known to accurately classify traffic signs, even reaching perfect classification performance under nominal operating conditions [15], [16], [17]. Unfortunately, the adverse environmental conditions, or the malfunctions of the camera, may produce low-quality frames that may negatively impact the performance of classifiers.…”

Preprocessing of traffic sign images based on improved histogram equalization and bilinear interpolation

“…Under severe weather conditions, visibility significantly decreases with varying levels of haze. ATIF M et al [4] used histogram equalization to enhance the contrast between the background and foreground.…”

Tolerate Failures of the Visual Camera With Robust Image Classifiers

“…We assessed this methodology by considering the deployment of image classifiers in autonomous vehicles for Traffic Sign Recognition (TSR), which may be significantly affected by visual camera failures [22], [28], [33]. We gathered three well-known TSR datasets, namely the German Traffic Sign Recognition Benchmark (GTSRB, [10]), the Belgium Traffic Sign (BelgiumTSC, [11]), and the Dataset of Italian Traffic Signs (DITS, [12]), applied AlexNet [7], MobileNetV2 [9], and Inceptionv3 [8] DNNs, which have wide application in the TSR domain [3], [4], [5], and injected a total of 13 visual camera failures under different configurations.…”