Relative performance of Bayesian clustering software for inferring population substructure and individual assignment at low levels of population differentiation

Abstract

Traditional methods for characterizing genetic differentiation among populations rely on a priori grouping of individuals. Bayesian clustering methods avoid this limitation by using linkage and Hardy-Weinberg disequilibrium to decompose a sample of individuals into genetically distinct groups. There are several software programs available for Bayesian clustering analyses, all of which describe a decrease in the ability to detect distinct clusters as levels of genetic differentiation among populations decrease. However, no study has yet compared the performance of such methods at low levels of population differentiation, which may be common in species where populations have experienced recent separation or high levels of gene flow. We used simulated data to evaluate the performance of three Bayesian clustering software programs, PAR-TITION, STRUCTURE, and BAPS, at levels of population differentiation below F ST =0.1. PARTITION was unable to correctly identify the number of subpopulations until levels of F ST reached around 0.09. Both STRUCTURE and BAPS performed very well at low levels of population differentiation, and were able to correctly identify the number of subpopulations at F ST around 0.03. The average proportion of an individual's genome assigned to its true population of origin increased with increasing F ST for both programs, reaching over 92% at an F ST of 0.05. The average number of misassignments (assignments to the incorrect subpopulation) continued to decrease as F ST increased, and when F ST was 0.05, fewer than 3% of individuals were misassigned using either program. Both STRUCTURE and BAPS worked extremely well for inferring the number of clusters when clusters were not well-differentiated (F ST =0.02-0.03), but our results suggest that F ST must be at least 0.05 to reach an assignment accuracy of greater than 97%.

Broad-scale genetic homogeneity in natural populations of common hazel (Corylus avellana) in Ireland

“…Ninety-five percent confidence intervals are indicated by dashed red lines. Note that in some correlograms, the first two distance intervals (0-50 and 50-100 m) may be at a different scale to subsequent intervals secondly, the BAPS algorithm has previously been shown to overestimate the number of clusters when levels of differentiation are low (Latch et al 2006). The observed weak pattern of IBD is also consistent with populations being well connected by gene flow in a stepping-stone fashion (Wright 1943;Slatkin 1993) and is comparable to that found in populations from Germany over a similar geographical scale (Leinemann et al 2013).…”

Regional genetic differentiation among populations of Cladocora caespitosa in the Western Mediterranean

“…The studied clusters share alleles and allelic frequencies and, therefore, are not fully isolated from each other. Even so, this analysis should be considered with caution as the performance of STRUCTURE algorithms decreases with sample size (Latch et al 2006), such as might have occurred for the population of Cape Palos (N = 13). Small but highly significant F ST values were found between the Columbretes Islands population and the others, while slight but significant or non-significant values were found among the other populations.…”

Historical isolation and hydrodynamically constrained gene flow in declining populations of the South-African abalone, Haliotis midae

“…The methods at present most frequently applied are model-based clustering analysis that do not assume a priori grouping of individuals into populations. The latter has greatly advanced the ability for detecting genetic discontinuity (Pritchard et al 2000;Latch et al 2006) while methods including spatial information are considered most effective in estimating the number of genetic units in conservation biology (Chen et al 2007;Segelbacher et al 2010). Also, a combination of different types of molecular markers could aid in defining integrated management units because of their ability to reveal different levels of genetic structuring (e.g.…”