Feature pyramid network with self-guided attention refinement module for crack segmentation

Abstract

Automated pavement crack segmentation is challenging due to the random shape of cracks, complex background textures and the presence of miscellaneous objects. In this paper, we implemented a Self-Guided Attention Refinement module and incorporated it on top of a Feature Pyramid Network (FPN) to model long-range contextual information. The module uses multi-scale features integrated from different layers in the FPN to refine the features at each layer of the FPN using a self-attention mechanism. The module enables the earlier layers and deeper layers of FPN to suppress noise and increase the crack details, respectively. The proposed network obtains an F1 score of 79.43% and 96.19% on the Crack500 and CFD datasets, respectively. Furthermore, the network also generalizes better than other state-of-the-art methods when tested on uncropped Crack500 and field images using the weights trained on CFD. In addition, ablation tests using the Crack500 dataset are conducted. The Self-Guided Attention Refinement module increases the average F1 score and recall by 0.6% and 0.8% while roughly maintaining the average precision. From the ablation test, the inclusion of the Self-Guided Attention Refinement module allows the network to segment finer and more continuous cracks. Then, the module is incorporated on PANet, DeepLab v3+ and U-Net to verify the improvements made to FPN. The results show that the module improves the F1 score, precision and recall compared to the absence of it. Moreover, the Self-Guided Attention Refinement Module is compared with the Self-Adaptive Sparse Transform Module (SASTM). The results show that the Self-Guided Attention Refinement Module provides a more consistent improvement.

Robust multitask compressive sampling via deep generative models for crack detection in structural health monitoring

“…More importantly, MT-GAN-CS achieves a high F 1 Score of 0.8 at CR = 128 even the used tasks are not similar to each other, such a performance level is comparable or superior to existing crack segmentation methods. [55][56][57] Furthermore, Figure 12 confirms that the reconstruction accuracy of MT-GAN-CS is basically the same as that of ST-GAN-CS, but its time cost to reconstruct one image block is only about a quarter of the cost by ST-GAN-CS (see Appendix D). Therefore, we conclude that MT-GAN-CS can speed up reconstruction without sacrificing the accuracy of recovered crack regions.…”

Fine‐grained crack segmentation for high‐resolution images via a multiscale cascaded network

“…Ong et al. (2023) proposed a multiscale encoder–decoder architecture with the embedment of self‐guided attention refinement modules, effectively suppressing background noise while enhancing network representation of crack details. Wu et al.…”

“…The authors have specifically noticed that among the previously discussed seven HR representation methods, multiscale sampling enhances the representation of small targets by leveraging multiscale feature information, while physical cascading operations achieve refined representation of randomly shaped targets by incorporating physical constraints. Notably, these two methods both exhibit relatively low model complexity, thus deeply mitigating the dependence on high GPU memory for HR image inference (Cheng et al., 2020; Ong et al., 2023). Consequently, this study aims to integrate these two methods to bridge the research gap in fine‐grained representation of HR crack images.…”

Encoder–decoder with pyramid region attention for pixel‐level pavement crack recognition

“…Inspired by NL, Wan et al (2021) designed CrackResAttentionNet for pavement crack detection based on the encoder-decoder network, where two self-attention-based attention modules were added after different encoder layers to aggregate long-range context information. Ong et al (2023) used selfattention to refine each feature pyramid network (FPN) layer so that the deep and shallow layers of the FPN could enhance crack information and reduce noise impact, respectively. However, the large computational cost of the self-attention mechanism severely limits the detection speed.…”

“…Ong et al. (2023) used self‐attention to refine each feature pyramid network (FPN) layer so that the deep and shallow layers of the FPN could enhance crack information and reduce noise impact, respectively. However, the large computational cost of the self‐attention mechanism severely limits the detection speed.…”