The electrocardiogram (ECG) is one of the most powerful tools used in hospitals to analyze the cardiovascular status and check health, a standard for detecting and diagnosing abnormal heart rhythms. In recent years, cardiovascular health has attracted much attention. However, traditional doctors’ consultations have disadvantages such as delayed diagnosis and high misdiagnosis rate, while cardiovascular diseases have the characteristics of early diagnosis, early treatment, and early recovery. Therefore, it is essential to reduce the misdiagnosis rate of heart disease. Our work is based on five different types of ECG arrhythmia classified according to the AAMI EC57 standard, namely, nonectopic, supraventricular ectopic, ventricular ectopic, fusion, and unknown beat. This paper proposed a high-accuracy ECG arrhythmia classification method based on convolutional neural network (CNN), which could accurately classify ECG signals. We evaluated the classification effect of this classification method on the supraventricular ectopic beat (SVEB) and ventricular ectopic beat (VEB) based on the MIT-BIH arrhythmia database. According to the results, the proposed method achieved 99.8% accuracy, 98.4% sensitivity, 99.9% specificity, and 98.5% positive prediction rate for detecting VEB. Detection of SVEB achieved 99.7% accuracy, 92.1% sensitivity, 99.9% specificity, and 96.8% positive prediction rate.