Oryza rufipogon Griff. is a valuable germplasm resource for rice genetic improvement. However, natural habitat loss has led to the erosion of the genetic diversity of wild rice populations. Genetic diversity analysis of O. rufipogon accessions and development of the core collection are crucial for conserving natural genetic diversity and providing novel traits for rice breeding. In the present study, we developed 1,592 SNPs by multiplex PCR and next-generation sequencing (NGS) technology and used them to genotype 998 O. rufipogon accessions from 14 agroclimatic zones in Guangdong and Hainan Provinces, China. These SNPs were mapped onto 12 chromosomes, and the average MAF value was 0.128 with a minimum of 0.01 and a maximum of 0.499. The O. rufipogon accessions were classified into ten groups. The mean Nei’s diversity index and Shannon–Wiener index (I) were 0.187 and 0.308, respectively, in all populations, indicating that O. rufipogon accessions had rich genetic diversity. There were also differences in the genetic diversity of O. rufipogon resources in the 14 regions. Hainan populations possessed higher levels of genetic diversity, whereas the Guangzhou population had lower levels of genetic diversity than did the other populations. Phylogenetic analysis revealed that the genetic relationship among the distribution sites of O. rufipogon was closely related to geographical location. Based on genetic distance, a core collection of 299 accessions captured more than 99% of the genetic variation in the germplasm. This study provides insights into O. rufipogon conservation, and the constructed core collection provides valuable resources for future research and genomics-assisted breeding of rice.