Archaeological dental calculus, or mineralized plaque, is a key tool to track the evolution of oral microbiota across time in response to processes that impacted our culture and biology, such as the rise of farming during the Neolithic. However, the extent to which the human oral flora changed from prehistory until present has remained elusive due to the scarcity of data on the microbiomes of prehistoric humans. Here, we present our reconstruction of oral microbiomes via shotgun metagenomics of dental calculus in 44 ancient foragers and farmers from two regions playing a pivotal role in the spread of farming across Europe—the Balkans and the Italian Peninsula. We show that the introduction of farming in Southern Europe did not alter significantly the oral microbiomes of local forager groups, and it was in particular associated with a higher abundance of the species Olsenella sp. oral taxon 807. The human oral environment in prehistory was dominated by a microbial species, Anaerolineaceae bacterium oral taxon 439, that diversified geographically. A Near Eastern lineage of this bacterial commensal dispersed with Neolithic farmers and replaced the variant present in the local foragers. Our findings also illustrate that major taxonomic shifts in human oral microbiome composition occurred after the Neolithic and that the functional profile of modern humans evolved in recent times to develop peculiar mechanisms of antibiotic resistance that were previously absent.