Image classification always has open challenges for computer vision research. Nowadays, deep learning has promoted the development of this field, especially in Convolutional Neural Networks (CNNs). This article proposes the development of efficiently scaled dilation of DropBlock optimization in CNNs for the fungus classification, which there are five species in this experiment. The proposed technique adjusts the convolution size at 35, 45, and 60 with the max-polling size 2 × 2. The CNNs models are also designed in 12 models with the different BlockSizes and KeepProp. The proposed techniques provide maximum accuracy of 98.30% for the training set. Moreover, three accurate models, called Precision, Recall, and F1-score, are employed to measure the testing set. The experiment results expose that the proposed models achieve to classify the fungus and provide an excellent accuracy compared with the previous techniques. Furthermore, the proposed techniques can reduce the CNNs structure layer, directly affecting resource and time computation.