This paper develops a new clustering technique based on Hellinger distance (HD) that can improve the Particle Swarm Optimization (PSO) search. Selection of features plays an important role in disease detection. Evidently, most machine learning algorithms are unable to accurately identify the optimal features in medical datasets due to the complexity of the data. The main objective of utilizing Hellinger distance is to partition dataset into two groups that are both highly similar and harmonious. Consequently, improve the accuracy of the proposed system by applying particle swarm optimization (PSO) to select more effective features. Technically, the extracted features from compiled MIT-BIH Arrhythmia dataset are applied to Minkowski classifier and several machine learning algorithms: KNN, DT, SVM, RF, and NB. The experiments demonstrated that the suggested model improves performance for classifying ECG signals by increasing accuracy, which reflects the importance of the modification made.