Anxi Tieguanyin belongs to the oolong tea category and is one of the top ten most famous teas in China. In this study, hyperspectral imaging (HSI) technology was combined with chemometric methods to achieve the rapid determination of free amino acid and tea polyphenol contents in Tieguanyin tea. Here, the spectral data of Tieguanyin tea samples of four quality grades were obtained via visible near-infrared hyperspectroscopy in the range of 400–1000 nm, and the free amino acid and tea polyphenol contents of the samples were detected. First derivative (1D), normalization (Nor), and Savitzky–Golay (SG) smoothing were utilized to preprocess the original spectrum. The characteristic wavelengths were extracted via principal component analysis (PCA), competitive adaptive reweighted sampling (CARS), and the successive projection algorithm (SPA). The contents of free amino acid and tea polyphenol in Tieguanyin tea were predicted by the back propagation (BP) neural network, partial least squares regression (PLSR), random forest (RF), and support vector machine (SVM). The results revealed that the free amino acid content of the clear-flavoured Tieguanyin was greater than that of the strong-flavoured type, that the tea polyphenol content of the strong-flavoured Tieguanyin was greater than that of the clear-flavoured type, and that the content of the first-grade product was greater than that of the second-grade product. The 1D preprocessing improved the resolution and sensitivity of the spectra. When using CARS, the number of wavelengths for free amino acids and tea polyphenols was reduced to 50 and 70, respectively. The combination of 1D and CARS is conducive to improving the accuracy of late modelling. The 1D-CARS-RF model had the highest accuracy in predicting the free amino acid (RP2 = 0.940, RMSEP = 0.032, and RPD = 4.446) and tea polyphenol contents (RP2 = 0.938, RMSEP = 0.334, and RPD = 4.474). The use of hyperspectral imaging combined with multiple algorithms can be used to achieve the fast and non-destructive prediction of free amino acid and tea polyphenol contents in Tieguanyin tea.