Sheep farming has been fundamental to many civilizations in the world and is practiced in India since antiquity. Several thousand years of adaptation to local environmental conditions and selective breeding have evolved 44 sheep breeds in India. They are paramount in terms of economic, scientific, and cultural heritage. Genetic characterization information is imperative for sustainable utilization and conservation of ovine heritage. In this study, the genetic diversity, differentiation, and structure of 11 indigenous sheep breeds from three different agro-ecological zones of India were explored with genomic microsatellite loci and mitochondrial DNA (D loop). The estimated diversity parameters indicated that populations retained high levels of genetic diversity (Na = 8.27 ± 0.17; Ho = 0.65 ± 0.01), which provides an optimistic viewpoint for their survival. However, significant inbreeding was also observed in the nine populations. Moderate genetic differentiation existed among the groups (FST = 0.129 ± 0.012), and most likely clusters existing in the dataset are seven. Phylogenetic clustering was in line with the geographical locations of sheep populations. Mitochondrial sequences revealed high haplotype diversity with the existence of maternal haplogroups A, B, and C, and signals of population expansion. Decreased genetic diversity and unique maternal lineage (C) in endangered Tibetan and Bonpala sheep breed, warrant their immediate scientific management. Overall, the quantitative data reported here on the extant variability, and genetic relationships among the Indian sheep breeds, provide critically important inputs that will be valuable for the decision-making process on their management, both for the conservation of endangered breeds, and formulation of breeding programs to check genetic erosion.