An aroma-based method for distinguishing different grades of Nongxiang Tieguanyin was explored by taking special grade (K110) and 1–4 grades (K101, K102, K103, and K104) of this tea as samples. Tea samples were analyzed by gas chromatography–mass spectrometry (GC-MS) combined with chemometrics. Results showed differences in the types and relative contents of aroma components among different grades of Nongxiang Tieguanyin tea. In the principal component analysis (PCA) scoring plot, except for K102 and K103, tea samples of different grades were distributed in different regions. Components satisfying variable important for the projection (VIP) > 1 and peak areas with significant differences (p < 0.05) among different tea grades were screened. Finally, 18 differential variables were screened out from 143 volatiles. The clustering results of these variables were consistent with those of PCA. K102 and K103 were initially clustered into one group and then clustered with K101, K110, and K104 in turn. The clear PCA separation of these samples and uniform hierarchical cluster analysis (HCA) clustering results suggests that GC-MS coupled with chemometrics analysis is a valid and accurate approach for discriminating different grades of Nongxiang Tieguanyin. The screened differential variables could represent a difference in aroma quality among five grades of Nongxiang Tieguanyin tea. Clear rules between peak area and the grade were also observed in some differential variables. 1-Ethylpyrrole and unknown-32 were positively correlated with grade. 2-Methylfuran, 2-ethylfuran, 2-methylidenecyclopentan-1-ol, mesityl oxide, 2-amylfuran, and D-limonene were negatively correlated with grade. The peak areas of methyl acetate, dimethyl sulfide, 6-methylhept-5-en-2-one, and (Z)-β-ocimene initially decreased but then increased with declining grade. The toluene content was especially high in K104 but only a negligible difference was observed among other grades. This study provides a potential method for differentiating Nongxiang Tieguanyin teas of different grades based on aroma. Unknown samples could be classified by comparison of their spatial distribution with those of known standard samples in PCA or HCA, as well as the peak area differences of differential variables between unknown samples and known standard samples.