To reveal the primary historical events that affected the geographic differentiation in freshwater organisms in the Japanese archipelago, we investigated the phylogeny and phylogeographic structure of the widely distributed freshwater fish Pseudogobio esocinus. We conducted comprehensive phylogeographic analyses based on partial sequences of the mitochondrial DNA cytochrome b (cyt b) gene in 1,212 P. esocinus specimens from 201 locations in 136 river systems along with sequence data from related species. We also examined the entire cyt b sequence and three nuclear gene sequences for a subset of individuals to estimate the phylogenetic relationships and divergence times. Historical processes shaping the present phylogeographic patterns were inferred by the Bayesian skyline plot, mismatch distribution and nested clade phylogeographic analyses. The mitochondrial and nuclear DNA phylogenies showed that the Japanese P. esocinus was not monophyletic and consisted of three largely differentiated groups. Divided by the Central Highlands, two groups were distributed in southwestern Japan, with a third group in northeastern Japan. The estimated divergence time indicated that the framework of the present ranges of the three groups was formed during or before the Early Pliocene. In southwestern Japan, the two groups often occurred sympatrically and included several regional subgroups bounded by major mountain systems uplifted in the early Pleistocene, while in the northeastern Japanese group, recent (since about 0.3 million years ago) range expansion from the northwestern area was inferred. The results suggest the importance of the uplifting of the Central Highlands and interconnection with the continental area in the distribution and divergence of P. esocinus, which may have also influenced regional heterogeneity in the Japanese freshwater fauna. The contrasting phylogeographic patterns between northeastern and southwestern Japan probably reflect their topographic and geohistorical differences, and imply differences in the formation process of freshwater fauna between the two regions.