This study aims to develop a high-speed and nondestructive mildewed rice grain detection method. First, a set of microscopic images of rice grains contaminated by Aspergillus niger, Penicillium citrinum, and Aspergillus cinerea are acquired to serve as samples, and the mildewed regions are marked. Then, three YOLO-v5 models for identifying regions of rice grain with contamination of Aspergillus niger, Penicillium citrinum, and Aspergillus cinerea in microscopic images are established. Finally, the relationship between the proportion of mildewed regions and the total number of colonies is analyzed. The results show that the proposed YOLO-v5 models achieve accuracy levels of 89.26%, 91.15%, and 90.19% when detecting mildewed regions with contamination of Aspergillus niger, Penicillium citrinum, and Aspergillus cinerea in the microscopic images of the verification set. The proportion of the mildewed region area of rice grain with contamination of Aspergillus niger/Penicillium citrinum/Aspergillus cinerea is logarithmically correlated with the logarithm of the total number of colonies (TVC). The corresponding determination coefficients are 0.7466, 0.7587, and 0.8148, respectively. This study provides a reference for future research on high-speed mildewed rice grain detection methods based on MCV technology.