Gene tree discord, simplex plots, and statistical tests under the coalescent

Abstract

A simple graphical device, the simplex plot of quartet concordance factors, is introduced to aid in the exploration of a collection of gene trees on a common set of taxa. A single plot summarizes all gene tree discord, and allows for visual comparison to the expected discord from the multispecies coalescent model (MSC) of incomplete lineage sorting on a species tree. A formal statistical procedure is described that can quantify the deviation from expectation for each subset of four taxa, suggesting when the data is not in accord with the MSC, and thus either gene tree inference error is substantial or a more complex model such as that on a network may be required. If the collection of gene trees appears to be in accord with the MSC, the plots may reveal when substantial incomplete lineage sorting is present and coalescent based species tree inference is preferred over concatenation approaches. Applications to both simulated and empirical multilocus data sets illustrate the insights provided.

MSCquartets 1.0: quartet methods for species trees and networks under the multispecies coalescent model in R

“…Using new methodology developed by Mitchell et al (2019), with applications discussed further by Allman et al (2020), a qcCF can be tested for fit to the MSC on 4-taxon unrooted topological species trees.…”

“…For example, if the four taxa {a, b, c, d} occur on m ≤ n of the gene trees, and the trees are fully resolved, then the counts m ab|cd , m ac|bd , m ad|bc of the 3 possible displayed topologies can be tabulated, where xy|zw denotes the binary quartet tree separating xy from zw. The resultant quartet count concordance factor qcCF = (m ab|cd , m ac|bd , m ad|bc ) or its normalization CF = 1 m qcCF , the empirical quartet concordance factor (Allman et al, 2020), are the data summaries of focus. The vector CF estimates the expectation eCF under the MSC of quartet frequencies displayed on gene trees.…”