Automatic Pavement Defect Detection and Classification Using RGB-Thermal Images Based on Hierarchical Residual Attention Network

Chen, Cheng; Chandra, Sindhu; Seo, Hyungjoon

doi:10.3390/s22155781

Cited by 10 publications

(5 citation statements)

References 40 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The results showed that the proposed method outperformed the intensity feature registration approach. The benefits of combining infrared and visible images for the interpretation of infrastructure defects were also reported on previous studies [15,59,60], highlighting the concept that the different targeted defects have enhanced texture and color characteristics registered in visible images, while the thermal data register important temperature differences related to faulty materials or components.…”

Section: Texture-based Segmentation For Improving Multi-modal Analysi...supporting

confidence: 65%

Texture Analysis to Enhance Drone-Based Multi-Modal Inspection of Structures

Nooralishahi

Ramos

Pozzer

et al. 2022

Drones

View full text Add to dashboard Cite

The drone-based multi-modal inspection of industrial structures is a relatively new field of research gaining interest among companies. Multi-modal inspection can significantly enhance data analysis and provide a more accurate assessment of the components’ operability and structural integrity, which can assist in avoiding data misinterpretation and providing a more comprehensive evaluation, which is one of the NDT4.0 objectives. This paper investigates the use of coupled thermal and visible images to enhance abnormality detection accuracy in drone-based multi-modal inspections. Four use cases are presented, introducing novel process pipelines for enhancing defect detection in different scenarios. The first use case presents a process pipeline to enhance the feature visibility on visible images using thermal images in pavement crack detection. The second use case proposes an abnormality classification method for surface and subsurface defects using both modalities and texture segmentation for piping inspections. The third use case introduces a process pipeline for road inspection using both modalities. A texture segmentation method is proposed to extract the pavement regions in thermal and visible images. Further, the combination of both modalities is used to detect surface and subsurface defects. The texture segmentation approach is employed for bridge inspection in the fourth use case to extract concrete surfaces in both modalities.

show abstract

Section: Texture-based Segmentation For Improving Multi-modal Analysi...supporting

confidence: 65%

Texture Analysis to Enhance Drone-Based Multi-Modal Inspection of Structures

Nooralishahi

Ramos

Pozzer

et al. 2022

Drones

View full text Add to dashboard Cite

show abstract

“…A basic CNN architecture includes a convolutional layer, a pooling layer, and a fully connected layer [9,10,12,13].…”

Section: Cnnmentioning

confidence: 99%

Prediction of rock mass class ahead of TBM excavation face by ML and DL algorithms with Bayesian TPE optimization and SHAP feature analysis

Chen

Seo

2023

Acta Geotech.

Self Cite

View full text Add to dashboard Cite

In this paper, field construction data from the Singapore Metro Line project were used to study the mapping relationship and establish the prediction model between TBM operation data and the ground condition ahead of the excavation face. The study presents a multi-classifier competition mechanism to construct ten separate classifiers, including logistic regression, support vector machine, random forest, extremely randomized trees, adaptive boosting machine, extreme gradient boosting (Xgboost), light gradient boosting (LightGBM), categorical boosting, long short-term memory and convolutional neural network. The acquired data were used to select 28 key TBM operating parameters by a correlation-based feature selection method, and the selected parameters in the stabilization phase after removing the outliers were calculated as the input to the classifier, and a relatively balanced training set was obtained by the synthetic minority oversampling technique. The hyperparameters of each classifier were optimized using tree Parzen estimator Bayesian optimization. The prediction results show that LightGBM presents the best results among ten different machine and deep learning algorithms with an accuracy of 96.22%, precision of 96.94%, recall of 97.33% and F1-score of 97.33%. In addition, the effect of the input parameters of the LightGBM model on the prediction accuracy of the model was analyzed using Shapley additive explanations, and the effect of sample imbalance on the prediction performance was discussed.

show abstract

“…Ibragimov (2022) conducted a study on detecting distress in the pavement through convolutional neural networks, and Jiang et al (2022) detected and segmented cracks in road pavements through a two-step deep learning approach [21,22]. Chandra et al (2022) studied the seasonal effect to detect complex pavement defects and Chen et al (2022) used thermal images to detect pavement damages [23,24]. Recently, studies on locating damage such as potholes through 3D data are also being conducted [25].…”

Section: Introductionmentioning

confidence: 99%

Automatic Damage Detection of Pavement through DarkNet Analysis of Digital, Infrared, and Multi-Spectral Dynamic Imaging Images

Seo,

Shi,

2024

Sensors

Self Cite

View full text Add to dashboard Cite

It is important to maintain the safety of road driving by automatically performing a series of processes to automatically measure and repair damage to the road pavement. However, road pavements include not only damages such as longitudinal cracks, transverse cracks, alligator cracks, and potholes, but also various elements such as manholes, road marks, oil marks, shadows, and joints. Therefore, in order to separate categories that exist in various road pavements, in this paper, 13,500 digital, IR, and MSX images were collected and nine categories were automatically classified by DarkNet. The DarkNet classification accuracies of digital images, IR images, and MSX images are 97.4%, 80.1%, and 91.1%, respectively. The MSX image is a enhanced image of the IR image and showed an average of 6% lower accuracy than the digital image but an average of 11% higher accuracy than the IR image. Therefore, MSX images can play a complementary role if DarkNet classification is performed together with digital images. In this paper, a method for detecting the directionality of each crack through a two-dimensional wavelet transform is presented, and this result can contribute to future research on detecting cracks in pavements.

show abstract

Automatic Pavement Defect Detection and Classification Using RGB-Thermal Images Based on Hierarchical Residual Attention Network

Cited by 10 publications

References 40 publications

Texture Analysis to Enhance Drone-Based Multi-Modal Inspection of Structures

Texture Analysis to Enhance Drone-Based Multi-Modal Inspection of Structures

Prediction of rock mass class ahead of TBM excavation face by ML and DL algorithms with Bayesian TPE optimization and SHAP feature analysis

Automatic Damage Detection of Pavement through DarkNet Analysis of Digital, Infrared, and Multi-Spectral Dynamic Imaging Images

Contact Info

Product

Resources

About