Multi-level feature fusion in densely connected deep-learning architecture and depth-first search for crack segmentation on images collected with smartphones

Abstract

Cracks are important signs of degradation in existing infrastructure systems. Automatic crack detection and segmentation plays a key role in developing smart infrastructure systems. However, this field has been challenging over the last decades due to irregular shape of the cracks and complex illumination conditions. This article proposes a novel deep-learning architecture for crack segmentation at pixel-level. In this architecture, one convolutional layer is densely connected to multiple other layers in a feed-forward fashion. Max pooling layers are used to reduce the dimensions of the features, and transposed convolution layers are used for multi-level feature fusion. A depth-first search–based algorithm is applied as post-processing tool to remove isolated pixels and improve the accuracy. The method is tested on two previously published data sets. It can reach 92.02%, 91.13%, and 91.58% for the first data set, and 92.17%, 91.61%, and 91.89% for the second data set for precision, recall, and F1 score, respectively. The performance of the proposed method outperforms other state-of-the-art methods. At the end of the article, the influence of feature fusion methods and transfer learning are also discussed.

Data-Driven Structural Health Monitoring and Damage Detection through Deep Learning: State-of-the-Art Review

“…pixel groups with similar color levels. The earlier generation of heuristic methods for vision-based crack detection in concrete structures was based on edge detection algorithms by applying filters, such as Roberts, Prewitt, Sobel, and LoG (in the spatial domain), or Butterworth and Gaussian (in the frequency domain) [101]. Figure 4 compares the results of applying different filters for detecting cracks on a concrete surface.…”

“…In general, major ML-based problems include three techniques: classification, localization, and segmentation. Figure 5 illustrates the frequent crack detection approaches: classification [25], object localization [23], and pixel-level segmentation [101]. Using the classification method, the dataset is labeled as cracked, non-cracked (sound).…”

“…For more precise classification, Dung and Anh [33] proposed semantic segmentation to also identify path and density,. The typical object detection models attempt to fit a bounding box around the ROI [85], and semantic segmentation methods [136] or pixel-level classification [101], should be used to precisely delineate damage level, shape, and location. For pavement crack detection problems, Zhang et al [27,137] proposed CrackNet, an efficient model based on R-CNNs.…”

“… Illustration of ( a ) crack and non-crack image classification, ( b ) object localization, and ( c ) pixel-level segmentation (open-access images from Yang et al [ 101 ]). …”

A Multi-Level Feature Fusion Network for Remote Sensing Image Segmentation

“…Semantic segmentation algorithms are generally divided into the following categories: multi-level feature fusion [ 6 , 7 , 8 ], atrous convolution [ 9 , 10 , 11 , 12 ], UNet type networks [ 13 , 14 , 15 ], and boundary optimization [ 16 , 17 , 18 ]. The multi-level feature fusion method is well known to contain more spatial information in low-level feature maps, whereas high-level feature maps are richer in semantic information.…”