Surface fatigue crack identification in steel box girder of bridges by a deep fusion convolutional neural network based on consumer-grade camera images

Abstract

This study conducts crack identification from real-world images containing complicated disturbance information (cracks, handwriting scripts, and background) inside steel box girders of bridges. Considering the multilevel and multi-scale features of the input images, a modified fusion convolutional neural network architecture is proposed. As input, 350 raw images are taken with a consumer-grade camera and divided into sub-images with resolution of 64 × 64 pixels (67,200 in total). A regular convolutional neural network structure is employed as baseline to demonstrate the accuracy benefits from the proposed fusion convolutional neural network structure. The confusion matrix is defined for prediction performance evaluation on the test set. A total of six additional entire raw images are used to investigate the robustness and feasibility of the proposed approach. A binary conversion process based on the optimal entropy threshold method is applied and closely followed to identify the crack pixels in the sub-images. The effect of the super-resolution inputs on accuracy is investigated. Results show that the trained modified fusion convolutional neural network can automatically detect the cracks, handwriting, and background from the raw images. The recognition errors of the fusion convolutional neural network in both the training and validation processes are smaller than those of the regular convolutional neural network. The super-resolution process hurts the general identification accuracy.

Automatic Tunnel Crack Detection Based on U-Net and a Convolutional Neural Network with Alternately Updated Clique

“…In addition to the above three evaluation indicators, there are also three general indicators in crack detection, precision, recall and F1-score [19,25,37]. The precision represents the proportion of actual crack pixels in the predicted crack pixels, and the recall represents the proportion of correctly predicted crack pixels in the real crack pixels.…”

Automatic seismic damage identification of reinforced concrete columns from images by a region-based deep convolutional neural network

“…27 Cracks are one of the most attractive subjects for damage identification. Restricted Boltzmann machine and CNNs are broadly used for crack identification in the surfaces of steel structures, [28][29][30] concrete structures, 31 pavement, 32,33 tunnel, 34,35 and railway. 36,37 Besides investigations on cracks, studies on other types of structural damages are also carried out.…”

Concrete crack detection with handwriting script interferences using faster region‐based convolutional neural network

“…Conventional image‐processing techniques (IPTs) mostly deal with digital images based directly on their pixel values, and from their images produce outputs such as the regions of interest (RoI) for defects (Y. Xu, Bao, Chen, Zuo, & Li, ). Studies of crack detection using such techniques have mostly focused on edge detection methods, image processing and pattern recognition concepts (Abdel‐Qader, Abudayyeh, & Kelly, ; Adhikari, Moselhi, & Bagchi, ; Jahanshahi, Masri, Padgett, & Sukhatme, ; Tong, Guo, Ling, & Yin, ; X. Xu & Zhang, ; C. Zhang & Elaksher, ).…”

“…Therefore, a more carefully configured deep neural network (DNN) is essential for solving the true variety of real‐world problems. For example, different DNNs have been proposed to identify cracks in images with disturbing background interference (F. C. Chen & Jahanshahi, ; Kang & Cha, ; R. Li, Yuan, Zhang, & Yuan, ; Liang, ; Y. Xu, Bao et al., ). In addition, the noising issues and the layer multiplexing scheme in DNNs have been discussed (Koziarski & Cyganek, ; Ortega‐Zamorano, Jerez, Gómez, & Franco, ).…”

“…Therefore, some studies were conducted to deal with complex background issues. For example, some algorithms were proposed for the identification of cracks with complicated background information (e.g., hand‐marked information) on a steel structure, which, however, could not predict the location of the cracks (Y. Xu, Bao et al., ; Y. Xu, Li et al., ). The extraction of cracks still required certain complex post processing techniques.…”