Bacterial and Fungal diseases may affect the yield of stone fruit and cause damage to the Chlorophyll synthesis process, which is crucial for tree growth and fruiting. However, due to their similar visual shot-hole symptoms, novice agriculturalists and ordinary farmers usually cannot identify and differentiate these two diseases. This work investigates and evaluates the use of machine learning for diagnosing these two diseases. It aims at paving the way toward creating a generic deep learning-based model that can be embedded in a mobile phone application or in a web service to provide a fast, reliable, and cheap diagnosis for plant diseases which help reduce the excessive, unnecessary, or improper use of pesticides, which can harm public health and the environment. The dataset consists of hundreds of samples collected from stone fruit farms in the north of Jordan under normal field conditions. The image features were extracted using a CNN algorithm that was pre-trained with millions of images, and the diseases were identified using three machine learning classification algorithms: (1) K-nearest neighbour (KNN); (2) Stochastic Gradient Descent (SGD); and (3) Random Forests (RF). The resulting models were evaluated using 10-fold crossvalidation, with CNN-KNN achieving the best AUC performance with a score of 98.5%. On the other hand, the CNN-SGD model performed best in Classification Accuracy (CA) with a score of 93.7%. The results shown in the Confusion Matrix, ROC, Lift, and Calibration curves also confirmed the validity and robustness of the constructed models.