Detecting Cracks and Spalling Automatically in Extreme Events by End-to-End Deep Learning Frameworks

Abstract

Abstract. In this paper, we develop and implement end-to-end deep learning approaches to automatically detect two important types of structural failures, cracks and spalling, of buildings and bridges in extreme events such as major earthquakes. A total of 2,229 images were annotated, and are used to train and validate three newly developed Mask Regional Convolutional Neural Networks (Mask R-CNNs). In addition, three sets of public images for different disasters were used to test the accuracy of these models. For detecting and marking these two types of structural failures, one of proposed methods can achieve an accuracy of 67.6% and 81.1%, respectively, on low- and high-resolution images collected from field investigations. The results demonstrate that it is feasible to use the proposed end-to-end method for automatically locating and segmenting the damage using 2D images which can help human experts in cases of disasters.

Engineering deep learning methods on automatic detection of damage in infrastructure due to extreme events

“…This test indicates that it is possible for end-to-end deep learning methods like the latest Mask R-CNNs to precisely detect cracks at various scale in large earthquake events.2. Spalling and crack detection with new variants of Mask R-CNN as an end-to-end method 39 : Since APANet and HRNet Mask R-CNNs worked quite well for crack detection, we added the spalling damage into the detection task to check whether this solution is more robust and applicable in field investigation. Image collection from Yang’s spalling dataset 6 and our in-house generated dataset were relabeled, resulting in a total of 2229 curated images for training and validation.…”

“…Training data for the latter are from the collection of Yang et al 6 and our own work. 39 There are two datasets for testing. The first one is a spalling dataset by Yeum et al, 10 in which there are 1000 images with a uniform size of 640×480.…”

“…2) Cascaded networks for spalling detection: For this test, the RestNet was trained in Task 3 of φ-Net dataset (see Table 1), and 1,178 images were prepared for training the U-Net to mark the spalling. Training data for the latter are from the collection of Yang et al 6 and our own work 39 . There are two datasets for testing.…”

“…In addition, the imbalance among the datasets training for damage localization is still hard to overcome. So, the deep learning models are easily distracted by these objects which appear like cracks and spalling in images such as: 1) wires or cables; 2) trees; 3) fences; 4) shadow; 5) edges of windows, buildings or other artifact objects 39 . Through refining a few parameters in the testing step, we also observed that the exact shapes and numbers of the damage can be precisely marked in some images.…”

Bridge inspection component registration for damage evolution

“…24-27 in their literature reviews have found significant work which demonstrates the utilization of machine learning techniques to detect structural damage in image data. Examples of these types of deep learning methods for images include the detection of concrete cracks, 28,[30][31][32][33][34][35][36] steel cracks, [37][38][39][40][41] corrosion, 29,32,40,[41][42][43] spalling, 32,[45][46][47][48][49][50][51][52][53] etc. However, between 2017 and 2021, we only found one instance of an image-based structural damage change detection algorithm for infrastructure inspection.…”