As one of the remaining species of Caprinae only found in Asia, serows (Capricornis) and their classification and conservation have received increasing attention in recent years. However, their evolutionary history and population dynamics are not yet clear. To shed light on these topics, we report the first near-complete ancient mitochondrial genomes from two serow sub-fossils (CADG839 and CADG946) dating to 8860 ± 30 years and 2450 ± 30 years, and incorporate the newly obtained mitogenomes into the dataset of living serows (18 complete mitochondrial genomes drawn from National Center for Biotechnology Information, NCBI) to investigate their relationships and evolution. Phylogenetic results support four clades of serows that can be further divided into five subclades, indicating higher genetic diversity than previously thought. Notably, our two ancient samples do not form a separate branch but belong to Capricornis sumatraensis clade A together with modern individuals, which suggests genetic continuity between ancient and modern serows. Furthermore, our results suggest that the maternal divergences of serows occurred at the beginning of the Pleistocene. Bayesian estimation indicates that the first divergence among all serows happened approximately 2.37 Ma (95% highest posterior density, HPD: 2.74–2.02 Ma) when Japanese serow (Capricornis crispus) appeared, while the last divergence occurred within the Sumatran serow (C. sumatraensis clade A and B) around 0.37–0.25 Ma. Additionally, we found the effective maternal population size of C. sumatraensis increased around 225–160 and 90–50 ka, then remained stable since 50 ka. Overall, our study provides new insights into serow phylogeny and evolutionary history.