Background: Cardiovascular diseases are recognized generally to be among the number one illnesscausing death across the globe. Predicting heart disease using a computer-aided technique makes it easier for medical practitioners to diagnose and thereby savinglives andreducingcosts. Feature selection has become an essential component for developing Machinelearning models. It chooses the most relevant features from the available dataset,thereby shortening the training period, making the model easier to train, improving generalization and decreasing overfitting without necessarily compromising the system’s accuracy. Aim:The purpose of this work is to design and build an optimal model forthe prediction of heart diseases,especially at an early stage by considering certain features that are most relevant forthe prediction without compromising the system’s accuracy. Method: The Cleveland UCI dataset with 303 instances wereused in trainingthe model and the findings showthat selectKBest is an effective tool in improving the prediction of heart diseases. The performance metrics Accuracy, Sensitivity, Precision were measured.Results: the study found that when hybridizing k-Nearest Neighbor Bagging, Decision TreeBagging, Gradient Boosting generated the highest accuracy of 90%, 85% and 88% respectively.