The Quaternary climate affected the present species richness and geographic distribution patterns of amphibians by limiting their activities during the glacial period. The present study examined the phylogenetic relationships of Japanese toads (Bufo japonicus and B. torrenticola) and the demography of each lineage from the past to the present based on mitochondrial sequences and ecological niche models. Japanese toads are a monophyletic group with two main clades (clades A and B). Clade A represents B. j. formosus, including three clades (clades A1, A2, and A3). Clade B contains three clades, two of which corresponded to B. j. japonicus (clades B1 and B2) and the other to B. torrenticola. Clade B2 and B. torrenticola made a sister group, and, thus, B. j. japonicus is paraphyletic. Clades A and B diverged in the late Miocene 5.7 million years ago (Mya) during the period when the Japanese archipelago was constructed. The earliest divergence between the three clades of clade A was estimated at 1.8 Mya. Clades A1 and A2 may have diverged at 0.8 Mya, resulting from the isolation in the multiple different refugia; however, the effects of the glacial climate on the divergence events of clade A3 are unclear. Divergences within clade B occurred from the late Pliocene to the early Pleistocene (3.2–2.2 Mya). Niche similarity between the parapatric clade in clade B (clades B1 and B2) indicated their allopatric divergence. It was suggested that niche segregation between B. japonicus and B. torrenticola contributed to a rapid adaptation of B. torrenticola for lotic breeding. All clade of Japanese toads retreated to each refugium at a low elevation in the glacial period, and effective population sizes increased to construct the current populations after the Last Glacial Maximum. Furthermore, we highlight the areas of climate stability from the last glacial maximum to the present that have served as the refugia of Japanese toads and, thus, affected their present distribution patterns.