Image Enhanced Mask R-CNN: A Deep Learning Pipeline with New Evaluation Measures for Wind Turbine Blade Defect Detection and Classification

Abstract

Demand for wind power has grown, and this has increased wind turbine blade (WTB) inspections and defect repairs. This paper empirically investigates the performance of state-of-the-art deep learning algorithms, namely, YOLOv3, YOLOv4, and Mask R-CNN for detecting and classifying defects by type. The paper proposes new performance evaluation measures suitable for defect detection tasks, and these are: Prediction Box Accuracy, Recognition Rate, and False Label Rate. Experiments were carried out using a dataset, provided by the industrial partner, that contains images from WTB inspections. Three variations of the dataset were constructed using different image augmentation settings. Results of the experiments revealed that on average, across all proposed evaluation measures, Mask R-CNN outperformed all other algorithms when transformation-based augmentations (i.e., rotation and flipping) were applied. In particular, when using the best dataset, the mean Weighted Average (mWA) values (i.e., mWA is the average of the proposed measures) achieved were: Mask R-CNN: 86.74%, YOLOv3: 70.08%, and YOLOv4: 78.28%. The paper also proposes a new defect detection pipeline, called Image Enhanced Mask R-CNN (IE Mask R-CNN), that includes the best combination of image enhancement and augmentation techniques for pre-processing the dataset, and a Mask R-CNN model tuned for the task of WTB defect detection and classification.

Image Defect Recognition Method Based on Deep Learning Network

“…e fully connected layer in the VGG16 model needs to be modified on the basis of the VGG16 model. In order to reduce the computational overhead, the number of neurons in the last two fully connected layers is reduced to 512 and 5, and the last fully connected layer corresponds to the five types of weld inspection data: porous, cracked, unfused, unperforated, and defect free [20]. e VGG16 model is first trained on the ImageNet source domain data set to obtain the weight parameters of convolutional layers C1-C4, pooling layers P1-P4, and fully connected layers FC6-FC8.…”

YOLOv4 Object Detection Model for Nondestructive Radiographic Testing in Aviation Maintenance Tasks

“…Some modifications to the existing one-stage neural network net are also highlighted. The two-stage network model is a more accurate architecture than the one-stage network model; even though R-CNN [16] and R-FCN [21] are faster networks, neither is more accurate or faster than the YOLOv4 model. Both the Faster R-CNN and the R-CNN models can take advantage of a better feature extractor but are less significant with the SSD and DetectNet algorithms.…”

WTBD-YOLOv8: An Improved Method for Wind Turbine Generator Defect Detection

“…Tong et al [10] improved the backbone network and feature fusion of Faster R-CNN, thereby enhancing the accuracy of blade damage detection. Zhang et al [11] proposed an image enhancement detection process based on Mask R-CNN, outperforming You Only Look Once version 3 (YOLOv3) and YOLOv4, and introduced new evaluation criteria. Zhang et al [12] developed the Mask-MRNet network for blade fault detection, which improved detection performance by combining Mask R-CNN-512 and MRNet, selecting DenseNet-121 as the optimal classifier.…”