Physical classification is the procedure adopted by the rice unloading, delivery, storage, and processing units for the commercial characterization of the quality of the grains. This step occurs mostly by the conventional method, which demands more time and specialized labor, and the results are subjective since the evaluation is visual. In order to make the operation faster, more accurate, and less dependent, non-destructive technologies and computational intelligence can be applied to characterize grain quality. Therefore, this study aimed to characterize and predict the quality of whole, processed rice grains, as well as classify any defects present. This was achieved by sampling from the upper and lower points of four silo dryers with capacities of up to 40,000 sacks. The grain samples had moisture contents of 16%, 17%, 18%, and 19% and were subjected to drying-aeration until reaching 12% moisture content (w.b.). Near-infrared spectroscopy technology and Machine Learning algorithm models (Artificial Neural Networks, decision tree algorithms Quinlan’s algorithm, Random Tree, REPTree, and Random Forest) were employed for this purpose. By analyzing Pearson’s correlation statistics, a strong negative correlation (R2 = 0.98) was found between moisture content and the yield of whole grains. Conversely, a strong positive correlation (R2 = 0.97) was observed between moisture content and classified physical defects across the various characterized physicochemical constituents. These findings indicate the effectiveness of near-infrared spectroscopy technology. The Random Tree model (RandT) successfully predicted the grain quality outcomes and is therefore recommended as the model of choice, obtained Pearson’s correlation coefficient (r = 0.96), mean absolute error (MAE = 0.017), and coefficient of determination (R2 = 0.92). The results obtained here reveal that the combination of near-infrared spectroscopy technology and Machine Learning algorithm models is an excellent non-destructive alternative to manual physical classification for characterizing the physicochemical quality of whole and defective rice grains.