India is the habitat of many freshwater fish species that make a significant contribution to endemism. However, the native species population is declining at a very fast rate due to pollution, habitat loss or degradation, new species introduction, hydrologic alterations, and overfishing. The minor carp, Labeo gonius, is one of the important fish species in the Indian river systems and holds enormous potential in culture fisheries. Therefore, the precise status of population structure is a prerequisite for the management of the species. In the present study, we assessed the potential of mtDNA ATPase 6/8 and D-loop regions as markers to determine the genetic diversity and phylogeography of L. gonius from Indian rivers. We have selected five major Indian rivers for investigation which could represent geographically distant populations of the studied fish. A total of 210 ATPase 6/8 sequences (842 bp) and 203 D-loop region sequences (683 bp) of L. gonius from the five major Indian rivers were submitted to NCBI. Results revealed that ATPase 6/8 has 17 haplotypes and 70 polymorphic sites with an overall haplotype diversity of 0.560 ± 0.036 and nucleotide diversity of 0.00347 ± 0.00. Furthermore, D-loop analysis revealed 58 haplotypes, 67 polymorphic sites, 0.954 ± 0.006 overall haplotype diversity, and 0.01914 ± 0.00 nucleotide diversity. The hierarchical AMOVA analysis indicated that a high proportion of the genetic variation (76.46% and 70.22% for the D-loop and ATPase 6/8 regions, respectively) was attributed to among-population variation under gene pool-1 (considering all the stocks from Ganga, Brahmaputra, Teesta, Narmada, and Mahanadi as a single pool). Furthermore, in gene pool-2, samples were divided into two groups: group 1 (Himalayan rivers)—Ganga, Brahmaputra, and Teesta; and group 2 (peninsular rivers)—Narmada and Mahanadi. Genetic variations in the D-loop and ATPase 6/8 regions were found among populations within groups at 70.65% and 53.48%, respectively. Haplotype network analysis revealed that there were significant differences in distributions and frequencies. The observed unique haplotypes might be useful for differentiating the stocks. In brief, the present study suggested that the D-loop and ATPase 6/8 regions are potential markers and represent a significant contribution to the genetic characterization of the minor carp L. gonius from major Indian rivers. Moreover, the L. gonius population along the east coast especially in the Himalayan rivers (Ganga, Brahmaputra, and Teesta river basin) is found to be genetically more diverse. The results would provide a scientific basis for the assessment of fisheries resources and management of this important species.