Identifying subtypes of neuropsychiatric disorders based on characteristics of their brain activity has tremendous potential to contribute to a better understanding of those disorders and to the development of new diagnostic and personalized treatment approaches. Many studies focused on neuropsychiatric disorders examine the interaction of brain networks over time using dynamic functional network connectivity (dFNC) extracted from resting-state functional magnetic resonance imaging data. Some of these studies involve the use of either deep learning classifiers or traditional clustering approaches, but usually not both. In this study, we present a novel approach for subtyping individuals with neuropsychiatric disorders within the context of schizophrenia (SZ). We train an explainable deep learning classifier to differentiate between dFNC data from individuals with SZ and controls, obtaining a test accuracy of 79%. We next make use of cross-validation to obtain robust average explanations for SZ training participants across folds, identifying 5 SZ subtypes that each differ from controls in a distinct manner and that have different degrees of symptom severity. These subtypes specifically differ from one another in their interaction between the visual network and the subcortical, sensorimotor, and auditory networks and between the cerebellar network and the cognitive control and subcortical networks. Additionally, there are statistically significant differences in negative symptom scores between the subtypes. It is our hope that the proposed novel subtyping approach will contribute to the improved understanding and characterization of SZ and other neuropsychiatric disorders.