Tsoongiodendron odorum Chun is a large evergreen tree in the Magnoliaceae family and an ancient relict species represented by small wild populations. It has excellent material quality, high ornamental value, and scientific significance. However, due to the complicated natural reproduction and notable habitat destruction, its wild populations must be urgently conserved. We used genotyping-by-sequencing to examine 17 natural populations of T. odorum in China, the species’ primary habitat, to better understand the genetic diversity of this species and use its germplasm resources. T. odorum had a very low level of genetic diversity; its mean values for Ho, He, Pi, and PIC were 0.175, 0.123, 0.160, and 0.053, respectively. With an average within-population Fst of 0.023 and an inter-population gene flow Nm of 10.918, population genetic variation was primarily found within populations, demonstrating minute genetic divergence between populations. The 17 natural populations of T. odorum were divided into two major categories: the Fujian populations in eastern China and the Jiangxi, Guangdong, Hunan, and Guangxi populations in central and western China. Our research contributes to the understanding of T. odorum’s genetic diversity and organization and offers a theoretical framework for the species’ conservation, breeding, and selection.