The increase in the internal temperature of high voltage electrical instruments is due to a variety of factors, particularly, contact problems; environmental factors; unbalanced loads; and cracks in the high voltage current transformers, voltage transformers, insulators, or terminal junctions. This increase in the internal temperature can cause unusual disturbances and damage to high voltage electrical equipment. Therefore, early prevention measures of thermal anomalies in equipment are necessary to prevent high voltage equipment failure that might shut down the whole grid system. In this article, we propose a novel non-destructive approach to defect analysis in high voltage equipment by taking advantage of the infrared thermography and the deep learning (DL) approach from the machine learning paradigm. The infrared images of the components were captured using the FLIR T630 without disturbing the operations of the power grid. In the first stage, rich features maps from the convolutional layers of the AlexNet pretrained model were extracted. After feature extraction, the random forest (RF) and support vector machines (SVM) were trained for learning of the defective and non-defective high voltage electrical equipment. In an experimental analysis, the RF optimally learned the separation between defective and non-defective equipment with greater than 96% accuracy, outperforming all the other comparative approaches for deep and nondeep features. The proposed approach based on the RF is reliable and shows its efficacy for fault detection in high voltage electrical equipment.