Breast cancer, if diagnosed and treated early, has a better chance of surviving. Many studies have shown that a larger number of ultrasound images are generated every day, and the number of radiologists able to analyze this medical data is very limited. This often results in misclassification of breast lesions, resulting in a high false-positive rate. In this article, we propose a computer-aided diagnosis (CAD) system that can automatically generate an optimized algorithm. To train machine learning, we employ 13 features out of 185 available. Five machine learning classifiers were used to classify malignant versus benign tumors. The experimental results revealed Bayesian optimization with a tree-structured Parzen estimator based on a machine learning classifier for 10-fold cross-validation. The LightGBM classifier performs better than the other four classifiers, achieving 99.86% accuracy, 100.0% precision, 99.60% recall, and 99.80% for the FI score.