Defects detection of insulators is crucial for the safe operation of power grid. A strategy of domain knowledge-assisted convolutional neural network is implemented for evaluating various depths and sizes of internal defects in insulating composite materials. A novel periodic-based 2D structuring method for ultrasonic signals is used to assist the CNN feature extraction process, leveraging the invariance of defect types with respect to the ultrasound sampling window and real background noise levels for data augmentation to enhance signal fidelity. Two supervised learning-based CNN models are trained to demonstrate the effectiveness of the proposed method. It is observed that the periodic-based 2D representation of ultrasonic signals facilitated superior performance of the 2DCNN compared to the 1DCNN using one-dimensional signals. In our strategy, the 2D ultrasonic signal can be interpreted as a feature map depicting the dependencies among different reflected echoes, as well as the intra- and inter-periodic variations of individual echoes. This domain-knowledge-compliant representation enhances the interpretability of the convolutional neural network. The results show that the trained 2DCNN achieved a defect recognition accuracy of 98.3% on unseen test sets and provided a relatively conservative accuracy estimate of 90% for defect-free samples, fully meeting the real industrial detection requirements to avoid misjudgment and missed judgment. Utilizing domain knowledge to assist neural networks effectively improves the quality of models required for industrial inspection.