4Our study aimed to develop an explanatory method for predicting Coronary Artery Disease (CAD) classification using spect images. As we all know, deep neural networks usually consist of many layers connected to each other through interlocking network nodes. Even if we check the classes and describe their relationships, it is difficult to understand entirely how active neural networks make predictions. Therefore, deep learning is still considered a "Black box". Existing XAI (eXplainable Artificial Intelligence) approach can provide insights into the inside of a Deep Learning model allowing for transparency and interpretation. Our previous research helps doctors diagnose the CAD of patients by developing deep learning models using a multi-stage transfer learning framework. The model achieved 0.955 accuracy, 0.932 AUC, 0.944 sensitivity, and 0.889 specificity, showing effective performance. Our dataset includes 218 SPECT images from 218 imported patients collected at 108 Hospital in Hanoi, Vietnam. In this paper, We propose an explainable Deep Learning framework using three popular XAI approaches: LIME, GradCam, and RISE. These XAI approaches are effective tools for interpreting the prediction of deep learning models. We evaluate the effectiveness of the interpretation by visualizing the explained regions and using improved deletion and insertion with a threshold limit suitable for Binary Classification. The experiment results show that our model effectively diagnoses CAD and provides medical interpretation. Furthermore, the proposed method for evaluating the deletion and insertion metrics is considered more efficient for binary classification than the traditional metrics.