Simultaneous pixel-level concrete defect detection and grouping using a fully convolutional model

Abstract

Deep learning techniques have attracted significant attention in the field of visual inspection of civil infrastructure systems recently. Currently, most deep learning-based visual inspection techniques utilize a convolutional neural network to recognize surface defects either by detecting a bounding box of each defect or classifying all pixels on an image without distinguishing between different defect instances. These outputs cannot be directly used for acquiring the geometric properties of each individual defect in an image, thus hindering the development of fully automated structural assessment techniques. In this study, a novel fully convolutional model is proposed for simultaneously detecting and grouping the image pixels for each individual defect on an image. The proposed model integrates an optimized mask subnet with a box-level detection network, where the former outputs a set of position-sensitive score maps for pixel-level defect detection and the latter predicts a bounding box for each defect to group the detected pixels. An image dataset containing three common types of concrete defects, crack, spalling and exposed rebar, is used for training and testing of the model. Results demonstrate that the proposed model is robust to various defect sizes and shapes and can achieve a mask-level mean average precision ( mAP) of 82.4% and a mean intersection over union ( mIoU) of 75.5%, with a processing speed of about 10 FPS at input image size of 576 × 576 when tested on an NVIDIA GeForce GTX 1060 GPU. Its performance is compared with the state-of-the-art instance segmentation network Mask R-CNN and the semantic segmentation network U-Net. The comparative studies show that the proposed model has a distinct defect boundary delineation capability and outperforms the Mask R-CNN and the U-Net in both accuracy and speed.

Improving Concrete Crack Segmentation Networks through CutMix Data Synthesis and Temporal Data Fusion

“…The described U-Net is trained and cross-validated using the labeled crack images extracted from three public datasets [ 31 , 32 , 33 ] and the dataset previously prepared by the authors based on the local bridge inspection images acquired from Hong Kong Highways Department [ 7 , 34 ]. In total, 540 crack images with their corresponding labels were collected.…”

“…In search of more reliable techniques, recent studies focus on utilizing the power of deep convolutional neural networks (CNNs) for crack detection and segmentation. These studies demonstrated that fully convolutional neural networks (FCNs) have remarkable outcomes for crack segmentation which is the task of crack prediction at a pixel level [ 5 , 6 , 7 , 8 , 9 , 10 ]. The great advantage of FCNs is that they can process raw inspection images and produce high-resolution predicted crack masks.…”

A convolutional neural network for pipe crack and leak detection in smart water network

“…The proposed method for pipe leak/crack early warning is built on the framework of a CNN. Convolution Neural Networks (CNNs) are powerful and versatile machine learning models, and have been widely used in visual recognition, 19 object detection, 20 and acoustic scene classification. 21 The CNN model does not require prior selection of acoustic features from wave files and is capable of learning useful features from the input data and embedding these in millions of parameters inside its deep network.…”

Workflow for Off-Site Bridge Inspection Using Automatic Damage Detection-Case Study of the Pahtajokk Bridge

“…In recent years, research has mainly been focused on damage detection based on ConvNets and remarkable progress has been made in areas such as classifying individual cracks from images and locating them [26][27][28] using a bounding box. However, the segmentation [29][30][31][32][33] and quantification [34][35][36][37] of defects has been little researched because the lack of labeled data makes it difficult to generalize training models across a wide range of defect shapes.…”