Crack-SAM: Crack Segmentation Using a Foundation Model

Abstract

Ensuring the structural integrity of pavements requires precise crack detection and evaluation. Manual inspections, although essential, are labour-intensive, time-consuming, and susceptible to errors, emphasizing the need for automated visual inspection techniques. This study presents an integrated approach to crack assessment by utilizing advanced visual models such as the Detectron2 model zoo and the Segment Anything Model (SAM) on Dataset A and Dataset B, which contain images from diverse locations with complex backgrounds and varying crack structures. Experiments were conducted using the Detectron2 model with four baseline configurations (mask_rcnn_R_50_FPN_3x, mask_rcnn_R_101_FPN_3x, fast_rcnn_R_50_FPN_3x, and fast_rcnn_R_101_FPN_3x), selected for their proven performance in object detection tasks and their ability to balance computational efficiency with high detection accuracy. Additionally, SAM was fine-tuned with three loss functions (Focal Loss, DiceCELoss, and DiceFocalLoss) chosen for their effectiveness in handling class imbalance and improving segmentation accuracy. Results demonstrate that SAM fine-tuned with DiceFocalLoss outperforms Detectron2 in crack segmentation, achieving mean intersection over union (MIoU) values of 0.69 for Dataset A and 0.59 for Dataset B. The integration of Detectron2 with fast_rcnn_R_101_FPN_3x as the baseline and SAM with DiceFocalLoss involves training the Detectron2 model to generate approximate bounding boxes around objects of interest, which are then used as prompts for the SAM model to produce segmentation masks, resulting in MIoU values of 0.83 for Dataset A and 0.75 for Dataset B. These findings represent significant advancements in crack identification methods, with substantial implications for improving highway maintenance practices.