Plant diseases have become a problem, as both the quality and quantity of agricultural products can be significantly reduced. The goal of the research is to detect the infection in the plant leaf so that it can be cured before spreading of disease to other plants. The aim of the research is to find out the severity of the leaf disease so that it can be cured based on the level of severity. Also, crop loss can be predicted by using the severity loss. For object localization, we utilize 10,640 tomato leaf images of various classes from the PlantVillage standard repository. As a baseline for future work, we propose a model based on the Mask R‐CNN architecture with ResNet‐50 as the backbone that effectively performs instance segmentation for these six diseases. The dataset is annotated by using the VGG annotator tool and this annotated dataset would have been used to train the ‘Mask R‐CNN model’ and the ResNet50 backbone, fine‐tuning the network's weights to accurately detect and segment diseased regions on leaves. The outcomes of the suggested model achieved an average accuracy of 91.3% by using multiple performance indicators like accuracy, precision and F1 score and Recall. Based on the outcome, the severity of the disease is being identified on a scale of 0, 1, 2 and 3.