Automatic Pixel‐Level Crack Detection and Measurement Using Fully Convolutional Network

Detecting and measuring the damage on historic glazed tiles plays an important role in the maintenance and protection of historic buildings. However, the current visual inspection method for identifying and assessing superficial damage on historic buildings is time and labor intensive. In this article, a novel two‐level object detection, segmentation, and measurement strategy for large‐scale structures based on a deep‐learning technique is proposed. The data in this study are from the roof images of the Palace Museum in China. The first level of the model, which is based on the Faster region‐based convolutional neural network (Faster R‐CNN), automatically detects and crops two types of glazed tile photographs from 100 roof images (2,488 × 3,264 pixels). The average precision values (AP) for roll roofing and pan tiles are 0.910 and 0.890, respectively. The cropped images are used to form a dataset for training a Mask R‐CNN model. The second level of the model, which is based on Mask R‐CNN, automatically segments and measures the damage based on the cropped historic tile images; the AP for the damage segmentation is 0.975. Based on Mask R‐CNN, the predicted pixel‐level damage segmentation result is used to quantitatively measure the morphological features of the damage, such as the damage topology, area, and ratio. To verify the performance of the proposed method, a comparative study was conducted with Mask R‐CNN and a fully convolutional network. This is the first attempt at employing a two‐level strategy to automatically detect, segment, and measure large‐scale superficial damage on historic buildings based on deep learning, and it achieved good results.

Section: Comparative Studymentioning

confidence: 99%

Autonomous damage segmentation and measurement of glazed tiles in historic buildings via deep learning

Wang

Zhao

Zou

et al. 2019

“…CNNs can enhance the object detection and classification capability of image processing techniques as they can learn the appropriate features automatically from the training data without the need of prior image processing or feature extraction. CNN was employed for crack segmentation by classifying each pixel as either damaged or intact (X. Yang, Li, et al., ; A. Zhang et al., ). To find the extent of a defect in a large image, a primitive approach is to do raster scanning of the image with a fixed‐size sliding window and then apply the trained CNN on each small window patch.…”

Section: Introductionmentioning

confidence: 99%

Concrete bridge surface damage detection using a single‐stage detector

Zhang

Chang

Jamshidi

2019

235

Early and timely detection of surface damages is important for maintaining the functionality, reliability, and safety of concrete bridges. Recent advancement in convolution neural network has enabled the development of deep learning‐based visual inspection techniques for detecting multiple structural damages. However, most deep learning‐based techniques are built on two‐stage, proposal‐driven detectors using less complex image data, which could be restricted for practical applications and possible integration within intelligent autonomous inspection systems. In this study, a faster, simpler single‐stage detector is proposed based on a real‐time object detection technique, You Only Look Once (YOLOv3), for detecting multiple concrete bridge damages. A field inspection images dataset labeled with four types of concrete damages (crack, pop‐out, spalling, and exposed rebar) is used for training and testing of YOLOv3. To enhance the detection accuracy, the original YOLOv3 is further improved by introducing a novel transfer learning method with fully pretrained weights from a geometrically similar dataset. Batch renormalization and focal loss are also incorporated to increase the accuracy. Testing results show that the improved YOLOv3 has a detection accuracy of up to 80% and 47% at the Intersection‐over‐Union (IoU) metrics of 0.5 and 0.75, respectively. It outperforms the original YOLOv3 and the two‐stage detector Faster Region‐based Convolutional Neural Network (Faster R‐CNN) with ResNet‐101, especially for the IoU metric of 0.75.

“…In addition, the noising issues and the layer multiplexing scheme in DNNs have been discussed (Koziarski & Cyganek, ; Ortega‐Zamorano, Jerez, Gómez, & Franco, ). Fully convolutional networks were proposed to detect concrete surface defects, which were good for defect localization at the pixel level (S. Li, Zhao, & Zhou, ; Xue & Li, ; X. Yang et al., ). Moreover, a sliding window size is difficult to define in CNNs, as any testing images that have different sizes in comparison with those used for network training will be difficult to process, which makes it challenging for the network to cope with different scenarios.…”

Section: Introductionmentioning

confidence: 99%

Concrete crack detection with handwriting script interferences using faster region‐based convolutional neural network

Deng

Lee

2019

168

The current bridge maintenance practice generally involves manual visual inspection, which is highly subjective and unreliable. A technique that can automatically detect defects, for example, surface cracks, is essential so that early warnings can be triggered to prevent disaster due to structural failure. In this study, to permit automatic identification of concrete cracks, an ad‐hoc faster region‐based convolutional neural network (faster R‐CNN) was applied to contaminated real‐world images taken from concrete bridges with complex backgrounds, including handwriting. A dataset of 5,009 cropped images was generated and labeled for two different objects, cracks and handwriting. The proposed network was then trained and tested using the generated image dataset. Four full‐scale images that contained complex disturbance information were used to assess the performance of the trained network. The results of this study demonstrate that faster R‐CNN can automatically locate crack from raw images, even with the presence of handwriting scripts. For comparative study, the proposed network is also compared with You Only Look Once v2 detection technique.