Non-destructive testing (NDT) of magnetic materials such as aviation parts is an indispensable part of the civil aviation maintenance industry. The NDT of such metal materials often uses magnetic particle inspection (MPI) technology. This paper proposes an improved DeepLabv3+ semantic segmentation algorithm for automatic defect detection of aviation ferromagnetic parts after MPI. In the network structure, lightweight MobileNetV2 is the backbone feature extraction network. The Dense Atrous Spatial Pyramid Pooling (DenseASPP) structure is used to strengthen feature extraction. The influence of three different DenseASPP structures on the recognition effect is compared in the experiment. At the same time, the decoder is further optimized. The experimental results show that the Ours-DeepLabv3+ network model can effectively for automatic defect detection of aviation ferromagnetic parts after MPI. The Precision, Recall, F1-score, and IOU are 81.64%, 83.12%, 82.37%, and 71.23%, respectively, which are 7.48%, 5.45%, 6.50%, and 10.1% higher than the original DeepLabv3+, and defect detail segmentation is more accurate. Compared with other semantic segmentation algorithms, this method can effectively improve the accuracy of defect detection of aviation ferromagnetic parts and meet the requirements of defect detection.