It is a hot spot in the field of computer application to diagnose complex brain diseases such as Asperger syndrome (AS) using machine learning technology. To identify AS patients and detect lesions, this paper proposes a novel clustering-evolutionary random support vector machine (SVM) ensemble (CERSVME) based on graph theory. Firstly, we extract graph theory indexes from the resting-state functional magnetic resonance imaging (fMRI) data as sample features and construct an ensemble learner by integrating multiple SVM classifiers. Secondly, the base learners with high redundancy and poor classification ability are deleted through clustering evolutions to improve the performance of the model. Then the CERSVME model is used to classify fMRI image of AS patients and healthy controls. According to the classification results, a multi-stage analysis scheme is designed to find the AS-related brain areas. We validate the proposed approach on 135 participants from autism brain imaging data exchange cohort. The highest accuracy reported by the CERSVME reaches 95.24%. More importantly, the diseased brain areas such as middle frontal gyrus, hippocampus and precuneus are found based on their contributions to classification performances of the CERSVME. Our study provides useful assistances for the clinical detection of patients with AS.