While solar energy holds great significance as a clean and sustainable energy source, photovoltaic panels serve as the linchpin of this energy conversion process. However, defects in these panels can adversely impact energy production, necessitating the rapid and effective detection of such faults. This study explores the potential of using infrared solar module images for the detection of photovoltaic panel defects through deep learning, which represents a crucial step toward enhancing the efficiency and sustainability of solar energy systems. A dataset comprising 20,000 images, derived from infrared solar modules, was utilized in this study, consisting of 12 classes: cell, cell-multi, cracking, diode, diode-multi, hot spot, hot spot-multi, no-anomaly, offline-module, shadowing, soiling, and vegetation. The methodology employed the exemplar Efficientb0 model. From the exemplar model, 17,000 features were selected using the NCA feature selector. Subsequently, classification was performed using an SVM classifier. The proposed method applied to a dataset consisting of 12 classes has yielded successful results in terms of accuracy, F1-score, precision, and sensitivity metrics. These results indicate average values of 93.93% accuracy, 89.82% F1-score, 91.50% precision, and 88.28% sensitivity, respectively. The proposed method in this study accurately classifies photovoltaic panel defects based on images of infrared solar modules.