Weather classification has become a significant challenge due to the unpredictable nature of climate conditions. For farmers, predicting the start of the rainy season is very important. This is because it is related to the cost factor that must be incurred, and also, the waiting time for the harvest has an effect if the weather is not supportive. Farmers also have to prepare seeds for the start of their farming. Therefore, farmers who start nurseries early in the rainy season will miss significant planting time. Based on these problems, this study uses a convolutional neural network (CNN) for weather classification using cloud imagery. CNN is shown to classify different spectro-temporal features of sound and is thus suitable for cloud image classification. We collect cloud image data using secondary data. Our model used a layer based on the convolution CNN architecture with a pooling layer and a solid layer as the output layer. The cloud dataset used is 1230 data divided into five classes, namely cloudy, foggy, rain, shine, and sunrise, which we use to train our model in research for the feature extraction process using DenseNet and VGG19. We use two types of classification, namely fully connected and Global Average Pooling (GAP). Our model can achieve a classification accuracy of 90.8% DenseNet-Fully Connected from our training process. From our testing process, our model can reach 95.7% using DenseNet-Fully Connected classification accuracy. Thus, the CNN model proved very accurate in classifying cloud images.