Broccoli is a plant that has many benefits. The flower parts of broccoli contain protein, calcium, vitamin A, vitamin C, and many more. However, in its cultivation, broccoli plants have obstacles such as the presence of pests and diseases that can affect production of broccoli. To avoid this, the authors build a model to identify diseases in broccoli through leaf images with a size of 128x128 pixels. The model is constructed to classify healthy leaves, and disease leaves using the image processing method that uses machine learning stages. There are several stages, including K-Means segmentation, colour feature extraction, and classification using SVM (Support Vector Machine) with RBF kernel and PSO (Particle Swarm Optimization) for reduce dimensionality data. The model that has been built compares the SVM model and the SVM-PSO model. It produces good accuracy in the training of 97.63% and testing accuracy of 94.48% for SVM-PSO and 85.82% for training, and 86.25% for testing in the SVM model. Therefore, this proposed model can produce good results in categorizing healthy and diseased leaves in broccoli.