Agricultural products need to be inspected for quality and safety, and the issue of safety of agricultural products caused by quality is frequently investigated. Safety testing should be carried out before agricultural products are consumed. The existing technologies for inspecting agricultural products are time-consuming and require complex operation, and there is motivation to develop a rapid, safe, and non-destructive inspection technology. In recent years, with the continuous progress of THz technology, THz spectral imaging, with the advantages of its unique characteristics, such as low energies, superior spatial resolution, and high sensitivity to water, has been recognized as an efficient and feasible identification tool, which has been widely used for the qualitative and quantitative analyses of agricultural production. In this paper, the current main performance achievements of the use of THz images are presented. In addition, recent advances in the application of THz spectral imaging technology for inspection of agricultural products are reviewed, including internal component detection, seed classification, pesticide residues detection, and foreign body and packaging inspection. Furthermore, machine learning methods applied in THz spectral imaging are discussed. Finally, the existing problems of THz spectral imaging technology are analyzed, and future research directions for THz spectral imaging technology are proposed. Recent rapid development of THz spectral imaging has demonstrated the advantages of THz radiation and its potential application in agricultural products. The rapid development of THz spectroscopic imaging combined with deep learning can be expected to have great potential for widespread application in the fields of agriculture and food engineering.