In this study, the genetic diversity and population structure of 4 wild ancient tea tree (Camellia taliensis) populations at different altitudes (2,050, 2,200, 2,350, and 2,500 m) in Qianjiazhai Nature Reserve, Zhenyuan country, Yunnan province, were investigated using EST-SSR molecular markers to compare their genetic variation against altitude. In total, 182 alleles were detected across all loci, ranging from 6 to 25. The top one informative SSR was CsEMS4 with polymorphism information content (PIC) of 0.96. The genetic diversity of this species was high, with 100% of loci being polymorphic, an average Nei’s gene diversity (H) of 0.82, and Shannon’s information index (I) of 1.99. By contrast, at the population level, the genetic diversity of wild ancient tea tree was relatively low, with values of H and I being 0.79 and 1.84, respectively. Analysis of molecular variance (AMOVA) revealed a minor genetic differentiation (12.84%) among populations, and most of the genetic variation (87.16%) was detected within populations. Using population structure analysis, we found that the germplasm of wild ancient tea tree was divided into three groups, and there was a substantial gene exchange among these three groups at different altitudes. Divergent habitats caused by altitudes and high gene flow played important roles in genetic diversity of wild ancient tea tree populations, which will provide new opportunities for promoting their protection and potential utilization.