In practice, early detection of disease is of high importance to practical value, corresponding measures can be taken at the early stage of plant disease. However, in the early stage of disease or when a rare disease occurs, there are limited training samples in practice, which makes machine learning especially deep learning models hardly work well while the stronger representative ability needs large-scale training data. To solve this problem, a fine grained-GAN based grape leaf spot identification method was proposed for local spot area image data augmentation to the generated local spot area images which were added and fed them into deep learning models for training to further strengthen the generalization ability of the classification models, which can effectively improve the accuracy and robustness of the prediction. Including 500 early-stage grape leaf spot images every category were fed into the proposed fine grained-GAN for local spot area data augmentation, 1000 local spot area sub images every category were generated in this study. The improved faster R-CNN was integrated in fine grained-GAN as local spot area detector. After that, the segmented sub-images and generated sub-images were mixed as training data input into the deep learning models, while the original segmented sub-images were used for testing. Experimental results showed that the proposed method had achieved higher identification accuracy on five state-of-art deep learning models; especially ResNet-50 had got 96.27% accuracy, which obtained significant improvements than other data augmentation methods and verified its satisfactory performance. This is of great practical significance for rare diseases or limited training samples.INDEX TERMS Fine grained-GAN, grape leaf spot identification, deep learning, few-shot learning, agricultural engineering.