The freshwater gudgeon Huigobio chenhsienensis (Cypriniformes: Gobionidae) is a small fish endemic to southern China. In this study, we used mitochondrial cytochrome b gene (Cytb), from wide-ranging samplings of H. chenhsienensis from the Ou River (the central of southern China) to the Yangtze River Basin (the northernmost part of southern China) to explore genetic variations and the evolutionary history of H. chenhsienensis in southern China. In total, 66 haplotypes were identified from Cytb sequences of 142 H. chenhsienensis individuals, which could be divided into lineages A, B, and C with divergence times of ~4.24 Ma and ~3.03 Ma. Lineage A was distributed in the lower reaches of the Yangtze River, the Oujiang River, and the Jiao River, lineage B was distributed in the Qiantang River and the Cao’e River, whereas lineage C was restricted to the Poyang Lake drainage from the middle reaches of the Yangtze River. Lineage A could be subdivided into sub-lineages A-I, A-II, A-III, and A-IV, with divergence times of 1.30, 0.97, and 0.44 Ma. Lineage C could be subdivided into sub-lineages C-I and C-II, with a divergence time of 0.85 Ma. Our findings indicate that climate change during the Pliocene and Pleistocene eras, as well as the limited dispersal ability of H. chenhsienensis, have been major drivers for shaping the phylogeographical patterns of H. chenhsienensis.