Pothole 3D Reconstruction With a Novel Imaging System and Structure From Motion Techniques

Abstract

Machine vision based evaluation systems are receiving increased attention, day by day, for automated quality inspection of roads. Industrial pavement scanners consist of laser scanners and are very expensive, hence inaccessible for everyone. The proposed work presents a simple and novel approach for 3D reconstruction of potholes for an automated inspection and road surface evaluation. The technique utilizes a Structure from Motion based 3D reconstruction algorithm, along with laser triangulation, to generate 3D point clouds of potholes. Alongside, a novel low-cost system, consisting of a single camera and a laser pointer, is also proposed. Keypoint matching techniques are employed, with the 5-point algorithm, on successive image frames to generate a point cloud. However, this point cloud is not metric yet, without scale information. The scale ambiguity is solved by making use of the laser pointer, and using the principle of triangulation. The laser spot is also detected in the same image sequence that is used for point-cloud building, cutting down the image capturing and processing overhead. The system has been benchmarked on artificial indentations with known dimensions, proving the robustness of the measurement scheme and hardware. Static and dynamic tests have been performed. The mean depth errors for measurement made by the imager statically and at dynamic speeds of 10 km/hr, 15 km/hr, and 20 km/hr are 5.3%, 7.9%, 14.4%, and 26.6%, whereas for perimeter the errors are 5.2%, 6.83 %, 11.8%, and 27.8%. The proposed, low-cost technique shows promising results in generating 3D point clouds for potholes.

Assessment of out‐of‐plane structural defects using parallel laser line scanning system

“…Second, in accordance with Ahmed et al. (2022), the point cloud coordinate system (i.e., $$O$$‐$$XYZ$$) is established at the reference surface as depicted in Figure 7. Here, $$X$$ and $$Y$$ represent the in‐plane coordinates of the reference plane, while $$Z$$ denotes the out‐of‐plane coordinate.…”

“…where 𝑋, 𝑌, and 𝑍 denote the point cloud coordinates from the two sets of laser inliers, and A 0 , B 0 , C 0 , and D 0 are physical coefficients that describe the equation. Second, in accordance with Ahmed et al (2022), the point cloud coordinate system (i.e., 𝑂-𝑋𝑌𝑍) is established at the reference surface as depicted in Figure 7. Here, 𝑋 and 𝑌 represent the in-plane coordinates of the reference plane, while 𝑍 denotes the out-of-plane coordinate.…”

Road potholes detection from MLS point clouds

“…The 3D reconstruction-based methods use ground cameras or unmanned aerial vehicles to acquire pavement images, and reconstruct the continuous pavement images into 3D point clouds by using methods such as binocular stereo vision [29,30] or structure from motion [31][32][33][34]. And then the potholes are detected based on the reconstructed 3D point cloud.…”

Enhanced pothole detection system using YOLOX algorithm

“…A lot of research has been published using different methods to automate the process of detecting potholes in the roads. Some of the commonly used pothole detection techniques are (i) vibration technique by using accelerometers [1,2] and other sensors [3] that can detect a bump in the road once the vehicle drives through it, (ii) 3D reconstruction technique [4,5] by using laser scanner method, stereo vision method or kinetic sensor method, (iii) im-age processing techniques [6,7] using GLCM (Gray-Level Co-Occurrence Matrix), RBF (Radial Basis Function) [6] and other morphological methods [8,9], and (iv) then the computer vision engineering approach using artificial intelligence, machine learning [10] and various deep learning techniques to detect potholes from a camera input in the form of an image or a video.…”

“…The second method is based on 3D reconstruction techniques [4,5]. These techniques include stereo vision based methods and laser based scanning methods.…”