This study uses clove imagery by classifying it according to ISO 2254-2004 standards: whole, headless, and mother clove. This type of clove will affect the quality and economic value when it has been dried. For this reason, it is necessary to take a first step to control cloves' quality. One way is to classify it from the start. This research will utilize the convolution neural network algorithm and compare it with model transfer learning and modified VGG16 architecture on clove images. In addition, research is also looking for the most optimal hyperparameter. The results of this study indicate that the application of convolution neural network (CNN) to clove images obtains an accuracy value of 84% using a hyperparameter of 50 epochs, a learning rate of 0.001, and a batch size of 16. Meanwhile, for the application of transfer learning VGG16, Resnet50, MobileNetV2, InceptionV3, DensetNet151, and modified VGG16 have respectively each of the highest accuracy including 95.70%, 76.15%, 96.89%, 98.07%, 98.96%, and 99.11%.