Many peninsulas in the temperate zone played an important role as refugia of various flora and fauna, and the southern Korean Peninsula also served as a refugium for many small mammals in East Asia during the Pleistocene. The Asian lesser white-toothed shrew, Crocidura shantungensis, is a widely distributed species in East Asia, and is an appropriate model organism for exploring the role of the Korean Peninsula as a refugium of small mammals. Here, we investigated phylogenetic relationships and genetic diversity based on the entire sequence of the mitochondrial cytochrome b gene (1140 bp). A Bayesian tree for 98 haplotypes detected in 228 C. shantungensis specimens from East Asia revealed the presence of three major groups with at least 5 subgroups. Most haplotypes were distributed according to their geographic proximity. Pairwise F's and analysis of molecular variance (AMOVA) revealed a high degree of genetic differentiation and variance among regions as well as among populations within region, implying little gene flow among local populations. Genetic evidence from South Korean islands, Jeju-do Island of South Korea, and Taiwan leads us to reject the hypothesis of recent population expansion. We observed unique island-type genetic characteristics consistent with geographic isolation and resultant genetic drift. Phylogeographic inference, together with estimates of genetic differentiation and diversity, suggest that the southern most part the Korean Peninsula, including offshore islands, played an important role as a refugium for C. shantungensis during the Pleistocene. However, the presence of several refugia on the mainland of northeast Asia is also proposed.