Research studies on the conservation genetics of endangered plants play a crucial role in establishing management plans for biodiversity conservation. Phoebe chekiangensis is a precious and scarce tree species resource in the East China region. To comprehend the origin, evolutionary history, geographical, and historical factors that has contributed to the current distribution pattern of Phoebe chekiangensis in the East China region, we conducted a phylogeographic analysis that utilized intergenic spacers of chloroplast DNA (cpDNA). We amplified and sequenced three spacer regions of cpDNA (psbC-trnS, trnL-Intro, and Ycf3) intergenic spacer regions of 306 individuals from 11 populations, encompassing the majority of its geographical range in China. Our analysis revealed a total of 11 haplotypes. The research findings show that the spacer regions of the cpDNA genetic diversity of Phoebe chekiangensis was Hd = 0.423, and the nucleotide diversity was Pi × 10−3 = 0.400. At the species level, the population differentiation index Fst = 0.25610 (p < 0.05), and the gene flow Nm = 0.73. The genetic variation between populations was 29.14%, while within populations, it was 70.86%, with the inter-population genetic variation much lower than the within-population variation. The divergence time between the genera Phoebe and Machilus was estimated to be approximately 37.87 mya (PP = 1; 95%HPD: 25.63–44.54 mya), and the crown group time of the genus Phoebe was estimated to be 21.30 mya (PP = 1; 95%HPD: 9.76–34.94 mya). The common ancestor of the 11 Phoebe chekiangensis haplotypes was 7.85 mya, while the H7, H8, and H10 haplotypes of Phoebe chekiangensis (northern region) differentiated relatively late, with a divergence time of 1.90 mya. Neutrality tests (NTs) and mismatch distribution analysis (MDA) suggest that the time frame for Phoebe chekiangensis to expand southwestward along Wuyishan was relatively short and its adaptability to the environment was low, thereby limiting the formation of new haplotypes. These results suggest that Phoebe chekiangensis exhibited greater adaptation to the northern subtropics than to the central subtropics, offering valuable insights for the conservation and utilization of germplasm resources.