Thermal defects of substation equipment have a great impact on the stability of power systems. Temperature is crucial for thermal defect detection in infrared images. The traditional detection methods, which have low efficiency and poor accuracy, record the temperature of infrared images manually. In this study, a thermal defect detection method based on infrared images using a convolutional neural network (CNN) is proposed. Firstly, the improved pre-processing method is applied to reduce background information, and the region of interest is located according to the contour and position information, hence improving the quality of images. Then, the temperature values are segmented to establish the dataset (T-IR11), which contains 11 labels. Finally, the CNN model is constructed to extract features, and the support vector machine is trained for classification. To verify the effectiveness of the proposed method, precision, recall, and F1 score are adopted and 10-fold cross-validation is employed on the T-IR11 dataset. The results demonstrate that the accuracy of the proposed method is 99.50%, and the performance is superior to that of previous methods in terms of infrared images. The proposed method can realize automatic temperature recognition and equipment with thermal defects can be recorded systematically, which has significant practical value for defect detection in substation equipment.