Deep neural networks can be used to distinguish partial discharge (PD) signals despite their complexity. This study analyzes the appropriateness of interpreting phase-resolved partial discharge (PRPD) signals using a convolutional neural network (CNN) through the Shapley additive explanation (SHAP) method. The generated PRPD signals were accumulated by applying AC voltage to four types of electrodes with a polyethylene sheet, followed by their conversion into scattered images to construct a classification model, CNN. The SHAP values for each pixel in the test images were then calculated. The result indicated that the pixels around the 0 V line retained high absolute SHAP values in every label, and the average of the summation of absolute SHAP values over all labels and all test images, which indicates the weight of each pixel, shows a similar tendency. Additionally, insight tests of the two CNN models were conducted, and the results showed that some structural defects could be detected by visualizing the SHAP values for each pixel. Finally, the verification of parameter-and-data vulnerability showed that SHAP has sufficient endurance against some types of instability in the data and model. Although the SHAP method lacks a perfect causal model because of its origin, the results imply that in appropriate use cases, weights on classifications of PD signals could be described by SHAP's interpretability.