The Siddis are a unique Indian tribe of African, South Asian, and European ancestry. While previous investigations have traced their ancestral origins to the Bantu populations from subSaharan Africa, the geographic localization of their ancestry has remained elusive. Here, we performed biogeographical analysis to delineate the ancestral origin of the Siddis employing an admixture based algorithm, Geographical Population Structure (GPS). We evaluated the Siddi genomes in reference to five African populations from the 1000 Genomes project, two Bantu groups from the Human Genome Diversity Panel (HGDP) and five South Indian populations. The Geographic Population Structure analysis localized the ancestral Siddis to Botsawana and its present-day northeastern border with Zimbabwe, overlapping with one of the principal areas of secondary Bantu settlement in southeast Africa. Our results further indicated that while the Siddi genomes are significantly diverged from that of the Bantus, they manifested the highest genomic proximity to the North-East Bantus and the Luhyas from Kenya. Our findings resonate with evidences supporting secondary Bantu dispersal routes that progressed southward from the east African Bantu center, in the interlacustrine region and likely brought the ancestral Siddis to settlement sites in south and southeastern Africa from where they were disseminated to India, by the Portuguese. We evaluated our results in the light of existing historical, linguistic and genetic evidences, to glean an improved resolution into the reconstruction of the distinctive population history of the Siddis, and advance our knowledge of the demographic factors that likely contributed to the contemporary Siddi genomes.