One of the main factors that contribute to a person's death throughout the world is cardiac disease nowadays. Hospitals have vast amounts of clinical data stored in biomedical devices and other systems. Understanding the facts and finding the most significant features that can increase the estimate accuracy that may cause heart disease is crucial. In this paper, a model has been proposed by using relevant characteristics of selected various feature selection, feature extraction, and machine learning classifiers. Filter and wrapper methods are used for feature selection, whereas principal component analysis (PCA) and linear discriminant analysis (LDA) are used for feature extraction. Both approaches were applied to the Cleveland heart disease dataset to get the relevant attribute subset. Once the attribute subset is selected, different machine learning classifiers, namely Logistic Regression (LR), Naïve Bayes (NB), Random Forest (RF), Extreme Gradient Boost (XGB), K-Nearest Neighbour (KNN), Decision Tree (DT), and Support Vector Classifier (SVC) were used to classify the presence and absence of heart disease. Finally, the top three classifiers with the most significant attributes and different feature selection/extraction techniques are discussed. The performance of the proposed model was also calculated using various performance evaluation metrics like accuracy, precision, recall and f1 score.