Quaternary climate oscillations and complex topography have tremendous effects on current distribution and genetic structure of species, and hence the Qinghai-Tibet Plateau (QTP), the largest plateau in the world, has become a hotspot for many phylogeographic studies. However, little is known about the phylogeographic pattern of herbaceous plants in QTP. Here, we investigate the genetic diversity, population structure and historical dynamics of Iris loczyi Kanitz, using five chloroplast DNA fragments and seven microsatellite markers. A total of fifteen populations, and 149 individuals were sampled throughout the QTP. High genetic diversity was detected both in cpDNA (Hd = 0.820) and SSR (Ho = 0.689, He = 0.699). Ten cpDNA haplotypes and 163 alleles were identified. AMOVA and clustering analyses revealed obvious differentiation between regions. The Nst, Gst and Mantel test showed significant phylogeographic structure of I. loczyi. The neutrality test and mismatch distribution analyses indicated that I. loczyi could not have undergone a historical population expansion, but population XS from the Qilian Mountain area could have experienced a local expansion. Bottleneck analyses indicated that I. loczyi had not experienced bottleneck recently. Based on cpDNA and SSR results, the Qilian Mountain area was inferred as a potential glacial refuge, and the southern Tibet valley was considered as a “microrefugia” for I. loczyi. These findings provided new insights into the location of glacial refuges for the species distributed in QTP, and supplemented more plant species data for the response of QTP species to the Quaternary climate.