Extensive DNA sequence data have made it possible to reconstruct human evolutionary history in unprecedented detail. We introduce a method to study the past several hundred thousand years. Our results show that (i) the Neanderthal-Denisovan lineage declined to a small size just after separating from the modern lineage, (ii) Neanderthals and Denisovans separated soon thereafter, and (iii) the subsequent Neanderthal population was large and deeply subdivided. They also (iv) support previous estimates of gene flow from Neanderthals into modern Eurasians. These results suggest an archaic human diaspora early in the Middle Pleistocene.human evolution | archaic admixture | introgression | Neanderthals | Denisovans A round 600 kya, Europe was invaded by large-brained hominins using Acheulean stone tools (1, 2). They were probably African immigrants, because similar fossils and tools occur earlier in Africa. They have been called archaic Homo sapiens, Homo heidelbergensis, and early Neanderthals, yet they remain mysterious. They may have been ancestors of Neanderthals and modern humans (3), or ancestors of Neanderthals only (4, 5), or an evolutionary dead end. According to this last hypothesis, they were replaced later in the Middle Pleistocene by a wave of African immigrants that separated Neanderthals from modern humans and introduced the Levallois stone tool tradition to Europe (6, 7). To address this controversy, we introduce a statistical method and use it to study genetic data of Africans, Eurasians, Neanderthals, and Denisovans.Our method extends an idea introduced by Reich et al. (8,9). Their "ABBA-BABA" statistics infer admixture from the frequency with which derived alleles are shared by pairs of samples. As we have shown (10), these estimators have large biases when populations receive gene flow from more than one source. The magnitudes of these biases depend on the sizes and separation times of ancestral populations. Our method avoids bias by estimating these parameters simultaneously.To accomplish this, our method uses an expanded dataset. ABBA-BABA statistics summarize allele sharing by pairs of samples. We extend this approach to include larger subsets, such as trios of samples, and to use all available subsets. This opens a rich and heretofore unused window into population history.
Nucleotide Site PatternsAlthough our method can accommodate complex models, we work here with a four-population model of history (Fig. 1A), which has broad empirical support (11,12). In this model, Neanderthals (N ) contribute genes to Eurasians (Y ) but not to Africans (X ). The model allows no gene flow from Denisovans (D), for reasons explained below. Combinations of uppercase letters, such as ND, refer to the population ancestral to N and D. Lowercase letters, such as n and d , refer to individual haploid genomes sampled from these populations.The gene tree describes how genes coalesce within the tree of populations. Fig. 1B illustrates one of many possible gene trees. Although closely linked nucleotide sites tend to share...