Grapes are a type of vine that belongs to the Vitaceae family and has many health benefits. There are dozens of grape varieties that are widespread in Indonesia. Grape varieties can be differentiated based on their various leaf shapes. At first glance, it might look the same. However, if you look at the shape and character of each leaf, grapes have different types and leaf variants. In recent years, various plant leaf classification methods based on deep learning have been proposed. This research uses a deep learning method with the Faster R-CNN ResNet-50 algorithm and uses pre-trained COCO weights to classify grape varieties through leaf images. For this purpose, a dataset of grape leaf images from five varieties was taken independently. Based on the tests that have been carried out, it shows that the improved network can effectively increase the efficiency of network operation. After testing four times ranging from 3,000 steps to 8,000 steps, the accuracy of recognizing leaf variations reached the highest level of 90.11% at 8,000 test steps with a loss of 0.134721. The results of this research show that the algorithm can classify types of grapes based on their leaves.