The freshness of rice reflects the time that has elapsed since it was harvested and the extent of deterioration in the quality of the rice that has occurred during storage. Therefore, it is crucial to detect the freshness of rice samples; here, we undertake that task using terahertz images and a modified VGG network. Terahertz imaging is non-destructive, permits molecular fingerprinting, and is low in energy consumption. Terahertz imaging technology uses terahertz rays to irradiate the sample and obtains a terahertz image of the sample by processing and analyzing the transmission and reflection spectra of the sample. Terahertz imaging technology has been widely used in applications related to material identification, medical diagnoses, quality detection of agricultural products, and safety inspections. In this paper, terahertz images of rice stored for various lengths of time were analyzed using a terahertz imaging system. Due to a large amount of data and inconspicuous features of the terahertz image, the traditional 1D-VGG network is relatively insufficient in computing power. Thus, it is not well suited to the extraction of features from within the images. To resolve this issue, the Inception-ResNet-A asymmetric convolution module in the Inception-ResNet-V2 network has great computing power,which is introduced into the VGG19 network structure. This proposed network is found to increase identification accuracy up to 99.8%. This work indicates that terahertz images combined with the modified 1D-VGG network represent an efficient and practical method for identifying rice freshness; this work thus has great potential for use as a tool for ensuring food quality and safety.