Polymorphisms in mitochondrial (mt) DNA and Y-chromosomes of seven socially and linguistically diverse castes and tribes of Eastern India were examined to determine their genetic relationships, their origin, and the influence of demographic factors on population structure. Samples from the Orissa Brahmin, Karan, Khandayat, Gope, Juang, Saora, and Paroja were analyzed for mtDNA hypervariable sequence (HVS) I and II, eight Y-chromosome short tandem repeats (Y-STRs), and lineage-defining mutations diagnostic for Indian- and Eurasian-specific haplogroups. Our results reveal that haplotype diversity and mean pairwise differences (MPD) was higher in caste groups of the region (>0.998, for both systems) compared to tribes (0.917-0.996 for Y-STRs, and 0.958-0.988 for mtDNA haplotypes). The majority of paternal lineages belong to the R1a1, O2a, and H haplogroups (62.7%), while 73.2% of maternal lineages comprise the Indian-specific M*, M5, M30, and R* mtDNA haplogroups, with a sporadic occurrence of West Eurasian lineages. Our study reveals that Orissa Brahmins (a higher caste population) have a genetic affinity with Indo-European speakers of Eastern Europe, although the Y-chromosome data show that the genetic distances of populations are not correlated to their position in the caste hierarchy. The high frequency of the O2a haplogroup and absence of East Asian-specific mtDNA lineages in the Juang and Saora suggest that a migration of Austro-Asiatic tribes to mainland India was exclusively male-mediated which occurred during the demographic expansion of Neolithic farmers in southern China. The phylogeographic analysis of mtDNA and Y-chromosomes revealed varied ancestral sources for the diverse genetic components of the populations of Eastern India.