Dalbergia odorifera T. Chen (Fabaceae) is a semi-deciduous tree species indigenous to Hainan Island in China. Due to its precious heartwood “Hualimu (Chinese)” and Chinese medicinal components “Jiangxiang”, D. odorifera is seriously threatened of long-term overexploitation and has been listed on the IUCN (International Union for Conservation of Nature’s) red list since 1998. Therefore, the elucidation of its genetic diversity is imperative for conservation and breeding purposes. In this study, we evaluated the genetic diversity of 42 wild D. odorifera trees from seven populations covering its whole native distribution. In total, 19 SSR (simple sequence repeat) markers harbored 54 alleles across the 42 samples, and the medium genetic diversity level was inferred by Nei’s gene diversity (0.36), observed (0.28) and expected heterozygosity (0.37). Among the seven wild populations, the expected heterozygosity (He) varied from 0.31 (HNQS) to 0.40 (HNCJ). The analysis of molecular variance (AMOVA) showed that only 3% genetic variation existed among populations. Moderate population differentiations among the investigated populations were indicated by pairwise Fst (0.042–0.115). Structure analysis suggested two clusters for the 42 samples. Moreover, the seven populations were clearly distinguished into two clusters from both the principal coordinate analysis (PCoA) and neighbor-joining (NJ) analysis. Populations from Haikou city (HNHK), Baisha autonomous county (HNBS), Ledong autonomous county (HNLD), and Dongfang city (HNDF) comprised cluster I, while cluster II comprised the populations from Wenchang city and Sansha city (HNQS), Changjiang autonomous county (HNCJ), and Wuzhisan city (HNWZS). The findings of this study provide a preliminary genetic basis for the conservation, management, and restoration of this endemic species.