Forestland fragmentation caused by overexploitation of forest resources can in principle reduce genetic diversity, limit gene flow and eventually lead to species developing strong genetic structure. However, the genetic consequences of recent anthropogenic fragmentation of tree species remain unclear. Taxus cuspidata, which has extremely small populations distributed mainly in Changbai Mt. in Northeast (NE) China, has recently endured severe habitat fragmentation. Here, we investigate the pattern of genetic diversity and structure, identify risk factors, predict the future distribution and finally provide guidelines for the conservation and management of this species. We used three chloroplast and two mitochondrial DNA fragments, which are both paternally inherited in yews but differ in mutation rates, to genotype a total of 265 individuals from 26 populations covering the distribution of the species in China. Both chloroplast and mitochondrial data showed high degrees of genetic diversity, extensive gene flow over the entire geographical range and historical stability of both effective population size and distribution of the species. However, ecological niche modeling suggests a decrease in suitable areas for this species by the years 2050 and 2070. The maintenance of high genetic diversity and the existence of sufficient gene flow suggest that recent fragmentation has not affected the genetic composition of the long-lived tree T. cuspidata. However, severe impacts of anthropogenic activities are already threatening the species. Conservation and management strategies should be implemented in order to protect the remnant populations.