The uneven distribution of light brightness on the surface of citrus spheres will make it difficult to directly partition the defect area of citrus surface. In view of the above shortcomings, a segmentation method of citrus surface defect based on brightness correction was proposed. In this method, Otsu algorithm was used to segment the background in HSI color space, and the mask template was obtained after the holes were filled. The mask template and I component were obtained by dot multiplication to obtain the I component image with the background removed. The I component image of the removed background was convolved with the established multi-scale Gaussian function to obtain the incident component image, and then the I component image after correction was obtained by point division with the incident component image. Finally, the single threshold method was used to extract citrus surface defects. The overall accuracy of this method is 94.7%, the recognition rate of normal fruit is 95%, and the recognition rate of defective fruit is 96.5%. It was found that the main reasons for misjudgment were that the color of the defect was similar to the normal fruit, the defect area was eliminated during denoising due to too small, and the convex or fold on the surface of the citrus was judged as the defect. This paper proposes an image brightness correction algorithm based on multi-scale Gaussian function, which can effectively correct citrus surface brightness and provide technical support for segmentation defects.