Herbaceous peony has been widely cultivated in China due to its substantial ornamental and medicinal value. In the present study, the phenotypic characteristics, total fatty acid (FA) content, and nine FA compositions of herbaceous peony seeds from 14 populations belonging to six species and one subspecies were determined by normal test and gas chromatography/mass spectrometry (GC/MS). The results showed that the phenotypic characteristics of seeds varied dramatically among species. The concentrations of five major FAs in seed oils were as follows: linoleic acid (173.95–236.51 μg/mg), linolenic acid (227.82–302.71 μg/mg), oleic acid (135.32–208.81 μg/mg), stearic acid (6.52–11.7 μg/mg), and palmitic acid (30.67–47.64 μg/mg). Correlation analysis demonstrated that oleic acid had the highest partial correlation coefficient with total FAs and might be applied to develop a model of phenotypic characteristics. FAs were significantly influenced by the following environmental factors: latitude, elevation, and annual average temperature. Based on the FA levels in the seed oils, clustering analysis divided 14 populations into two clusters. It was found that the average contents of oleic acid, linoleic acid, and total FAs in cluster I (147.16 μg/mg, 200.31 μg/mg, and 671.24 μg/mg, respectively) were significantly lower than those in cluster II (196.65 μg/mg, 220.16 μg/mg, and 741.78 μg/mg, respectively). Cluster I was perfectly consistent with subsect. Foliolatae, while cluster II was in good agreement with subsect. Dissectifoliae. Therefore, the FA composition of wild herbaceous peony seed oil might be used as a chemotaxonomic marker.