PotholeEye+: Deep-Learning Based Pavement Distress Detection System toward Smart Maintenance

Abstract

We propose a mobile system, called PotholeEye + , for automatically monitoring the surface of a roadway and detecting the pavement distress in real-time through analysis of a video. PotholeEye + pre-processes the images, extracts features, and classifies the distress into a variety of types, while the road manager is driving. Every day for a year, we have tested PotholeEye + on real highway involving real settings, a camera, a mini computer, a GPS receiver, and so on. Consequently, PotholeEye + detected the pavement distress with accuracy of 92%, precision of 87% and recall 74% averagely during driving at an average speed of 110 km/h on a real highway.

Integration of Smart Pavement Data with Decision Support Systems: A Systematic Review

“…In this analysis, this category considered vehicles equipped mainly with low-cost sensors and used in frequent monitoring actions, as opposed to the conventional vehicles used by infrastructure agencies, equipped with high-accuracy and expensive sensors which monitor the network periodically. The works that resorted to these smart probe vehicles focus either on image-based collection methods [28][29][30]32,36], or accelerometer, microphones and global positioning system (GPS) sensors for international roughness index (IRI) estimation and crack detection [28,31].…”

“…Since this analysis is usually carried out in a two-fold action, starting with the identification of surface distresses followed by the determination of quality indexes, the functional category was sub-divided into a second level. This sub-division aimed to distinguish the data analysis methods that focused on the identification and classification of surface distresses, such as superficial cracks [28][29][30]33,40,[62][63][64][65][66][67], potholes [25,32,36], patches [18], and others [37,41], and the estimation of pavement quality indexes, such as IRI [17,19], PCI [16] and other indexes proposed by some researchers [20,22,23,27,68]. These sub-categories were, in turn, divided into the type of adopted approach, either image processing or data-driven models for the case of identification and classification of surface distresses, and model-driven or data-driven for the estimation of pavement quality indexes.…”

“…Image processing methods and techniques are mostly used for (but not limited to) the detection of surface distresses, such as cracks and rutting [28][29][30]33,37,41,[62][63][64][66][67][68], potholes [25,32,36,37,41,68] and patches [18,37]. As mentioned in the previous section, image-based collection methods, which usually resort to high resolution cameras, have been widely studied.…”

“…As mentioned in the previous section, image-based collection methods, which usually resort to high resolution cameras, have been widely studied. The images captured by those cameras can be processed as twodimensional (2D) images for distress detection [18,25,30,32,33,36,37,41,[62][63][64][65][66]68], or used to generate 3D models of the pavement, aiming to assess the surface roughness [61], deformation levels [40] or patch depth [32]. A common step among image processing methods is image segmentation, which involves dividing the image in smaller sections based on certain features and characteristics, culminating in the isolation of the pavement distress that is intended to be detected.…”

“…Convolutional neural networks (CNN) are well-known machine learning methods usually applied in image detection systems, such as these ones. Hence, the majority of the systems described herein applied variations of CNN [25,32,33,36,[63][64][65][66][67], such as YOLO (you only look once) [41,68] or U-NET [37,68] architectures. Apart from CNN, though not as common, random forest [18] and support vector machine (SVM) [30,62] machine learning methods were also successfully used for distress detection.…”