Several machine learning algorithms have been proposed in recent years to design accurate classification systems for a wide range of diseases such as cancers, hepatitis, and coronavirus. In this study, the Classification and Regression Tree (CART) is proposed to predict breast cancer in the early stage, later applied to real data collected from the Sebha oncology center. The study focuses on improving the CART accuracy through several methods: (1) cross-validation, (2) dimensionality reduction and (3) hyper-parameter tuning. However, two crossvalidation strategies have been investigated namely: The K fold and stratified fold, followed by dimensionality reduction to determine the most effective features using two methods, namely: recursive feature elimination with cross-validation and principal component analysis, and lastly, investigating the most optimal CART parameters using two optimization algorithms, namely: grid search, and random search. The experimental results have shown that the best CART model which achieved 97% accuracy uses a stratified fold as a cross-validation strategy, recursive feature elimination with cross-validation as dimensionality reduction, and grid search as parameters tuning algorithm. Moreover, when compared to the original CART, the accuracy of the proposed CART has improved from 63% to 97%.