The adulteration in Chinese chestnuts affects the quality, taste, and brand value. The objective of this study was to explore the feasibility of the hyperspectral imaging (HSI) technique to determine the geographical origin of Chinese chestnuts. An HSI system in spectral range of 400–1000 nm was applied to identify a total of 417 Chinese chestnuts from three different geographical origins. Principal component analysis (PCA) was preliminarily used to investigate the differences of average spectra of the samples from different geographical origins. A deep-learning-based model (1D-CNN, one-dimensional convolutional neural network) was developed first, and then the model based on full spectra and optimal wavelengths were established for various machine learning methods, including partial least squares-discriminant analysis (PLS-DA) and particle swarm optimization-support vector machine (PSO-SVM). The optimal results based on full spectra for 1D-CNN, PLS-DA, and PSO-SVM models were 97.12%, 97.12%, and 95.68%, respectively. Competitive adaptive reweighted sampling (CARS) and a successive projections algorithm (SPA) were individually utilized for wavelengths selection, and the results of simplified models generally improved. The contrasting results demonstrated that the prediction accuracies of SPA-PLS-DA and 1D-CNN both reached 97.12%, but 1D-CNN presented a higher Kappa coefficient value than SPA-PLS-DA. Meanwhile, the sensitivities and specificities of SPA-PLS-DA and 1D-CNN models were both above 90% for the samples from each geographical origin. These results indicated that both SPA-PLS-DA and 1D-CNN models combined with HSI have great potential for the geographical origin identification of Chinese chestnuts.