Rice is one of the most important crops for food supply, and there are multiple differences in the quality of rice in different geographic regions, which have a significant impact on subsequent yields and economic benefits. The traditional rice identification methods are time-consuming, inefficient, and delicate. This study proposes a deep learning-based method for fast and non-destructive classification of rice grown in different geographic environments. The experiment collected rice with the name of Ji-Japonica 830 from 10 different regions, and a total of 10,600 rice grains were obtained, and the fronts and backsides of the seeds were photographed with a camera in batches, and a total of 30,000 images were obtained by preprocessing the data. The proposed improved residual network architecture, High-precision Residual Network (HResNet), was used to compare the performance of the models. The results showed that HResNet obtained the highest classification accuracy result of 95.13%, which is an improvement of 7.56% accuracy with respect to the original model, and validation showed that HResNet achieves a 98.7% accuracy in the identification of rice grown in different soil classes. The experimental results show that the proposed network model can effectively recognize and classify rice grown in different soil categories. It can provide a reference for the identification of other crops and can be applied for consumer and food industry use.