Developmental genetics research on mice provides a relatively sound understanding of the genes necessary and sufficient to make mammalian teeth. However, mouse dentitions are highly derived compared with human dentitions, complicating the application of these insights to human biology. We used quantitative genetic analyses of data from living nonhuman primates and extensive osteological and paleontological collections to refine our assessment of dental phenotypes so that they better represent how the underlying genetic mechanisms actually influence anatomical variation. We identify ratios that better characterize the output of two dental genetic patterning mechanisms for primate dentitions. These two newly defined phenotypes are heritable with no measurable pleiotropic effects. When we consider how these two phenotypes vary across neontological and paleontological datasets, we find that the major Middle Miocene taxonomic shift in primate diversity is characterized by a shift in these two genetic outputs. Our results build on the mouse model by combining quantitative genetics and paleontology, and thereby elucidate how genetic mechanisms likely underlie major events in primate evolution.paleontology | quantitative genetics | primates | neontology | dental variation T he relationship between genotype and phenotype is critical to evolutionary biology, because it influences how phenotypes respond to selective pressures and evolve (1, 2). Paleontologists have long sought to incorporate the etiology of the dental phenotype to inform on questions of environmental, dietary, and adaptive change over time (3)(4)(5)(6). This research has been advanced significantly by the revolution in developmental genetics over the past few decades (7). Experimental research on mice has yielded tremendous biological insight (8). However, for human phenotypes, ranging from inflammation (9) to placentation (10), the limitations of the mouse model due to the ∼140 million years ago of evolution that have occurred since our last common ancestor ∼70 Ma (11) are starting to be recognized. Here, we demonstrate how to overcome the limitations of the mouse model's application to the primate dentition by integrating research from quantitative genetics, neontology, and paleontology. The insights gained from this transdisciplinary approach have implications for all of these seemingly disparate subdisciplines of biology.

SignificanceThe frequency of the human-specific EDAR V370A isoform is highly elevated in North and East Asian populations. The gene is known to have several pleiotropic effects, among which are sweat gland density and ductal branching in the mammary gland. The former has led some geneticists to argue that the near-fixation of this allele was caused by selection for modulation of thermoregulatory sweating. We provide an alternative hypothesis, that selection instead acted on the allele’s effect of increasing ductal branching in the mammary gland, thereby amplifying the transfer of critical nutrients to infants via mother’s milk. This is likely to have occurred during the Last Glacial Maximum when a human population was genetically isolated in the high-latitude environment of the Beringia.

No abstract