Leaf diseases include various diseases like abscission, gummosis, Alternaria solani, vesicatoria, and Pucciniales. Farmers from all around the world are constantly concerned about the health of various plants. Our proposed technology identifies various leaf diseases at an early stage, alerting farmers and allowing them to take necessary control measures. The collection contains 25,362 photos of normal leaves, bacterial spot, rust, early blight, and other disease‐infected leaves in Solanum tuberosum, Solanum lycopersicum, Malus, Zea mays, and Piper nigrum. In this study, a transfer learning ResNet 50 algorithm and a mask regional convolutional neural network are used to segment leaf diseases. An improved faster mask regional‐convolutional neural network (IFMR‐CNN) method is proposed to find the affected areas in the plant. The experiment first gathers photos of leaf disease for preprocessing, then utilizes a VGG annotator to create labels for the data sets, which are split into a test set and training set. The proposed IFMR‐CNN approach is capable of localizing and classifying the disease with 96% accuracy. Accuracy is calculated based on the proportion of properly identified leaf samples in a dataset out of all the leaf samples. Accuracy of disease localization in plant leaves can be measured by the percentage of correctly identified regions of interest in the leaf image that contain the disease symptoms.