Genome scan methods against more complex models: when and how much should we trust them?

Abstract

The recent availability of next-generation sequencing (NGS) has made possible the use of dense genetic markers to identify regions of the genome that may be under the influence of selection. Several statistical methods have been developed recently for this purpose. Here, we present the results of an individual-based simulation study investigating the power and error rate of popular or recent genome scan methods: linear regression, Bayescan, BayEnv and LFMM. Contrary to previous studies, we focus on complex, hierarchical population structure and on polygenic selection. Additionally, we use a false discovery rate (FDR)-based framework, which provides an unified testing framework across frequentist and Bayesian methods. Finally, we investigate the influence of population allele frequencies versus individual genotype data specification for LFMM and the linear regression. The relative ranking between the methods is impacted by the consideration of polygenic selection, compared to a monogenic scenario. For strongly hierarchical scenarios with confounding effects between demography and environmental variables, the power of the methods can be very low. Except for one scenario, Bayescan exhibited moderate power and error rate. BayEnv performance was good under nonhierarchical scenarios, while LFMM provided the best compromise between power and error rate across scenarios. We found that it is possible to greatly reduce error rates by considering the results of all three methods when identifying outlier loci.

The challenge of separating signatures of local adaptation from those of isolation by distance and colonization history: The case of two white pines

“…Although both methods essentially operate based on the same principles, that is, they test for GEAs after controlling for the portion of variation that is due to neutral population structure, their results differed greatly ( P. strobus : 19% overlap; P. monticola : no overlap). The relative performance of Bayenv and LFMM depends on the demographic scenario and sampling design, and a relatively low overlap between the two methods has previously been observed in simulation studies (Lotterhos & Whitlock, 2015; de Villemereuil et al., 2014). …”

“…This method uses a hierarchical Bayesian mixed model based on a variant of PCA, in which neutral population structure is introduced via ( k ) unobserved or latent factors. We implemented the LFMM method using the default individual‐based data specification to avoid potential biases due to unequal population sample sizes (de Villemereuil et al., 2014). To determine k , we performed a PCA on individual allele frequencies using the LEA package in R (Frichot & François, 2015).…”

“…These methods show variable performances under different demographic scenarios (Excoffier et al., 2009; Frichot, Schoville, de Villemereuil, Gaggiotti, & François, 2015; Lotterhos & Whitlock, 2014, 2015; de Villemereuil, Frichot, Bazin, François, & Gaggiotti, 2014). Specifically, GEA methods have low power and high rates of false positives when environmental gradients are correlated with the main axes of neutral population structure (De Mita et al., 2013; Lotterhos & Whitlock, 2015; de Villemereuil et al., 2014).…”

“…These methods show variable performances under different demographic scenarios (Excoffier et al., 2009; Frichot, Schoville, de Villemereuil, Gaggiotti, & François, 2015; Lotterhos & Whitlock, 2014, 2015; de Villemereuil, Frichot, Bazin, François, & Gaggiotti, 2014). Specifically, GEA methods have low power and high rates of false positives when environmental gradients are correlated with the main axes of neutral population structure (De Mita et al., 2013; Lotterhos & Whitlock, 2015; de Villemereuil et al., 2014). Despite the fact that GEA methods have variable performances in such scenarios, many studies only report results from a single method, and very few report the degree of collinearity between environmental gradients and geography (e.g., Lee & Mitchell‐Olds, 2011) or phylogeographic lineages (e.g., Jaramillo‐Correa et al., 2015).…”

Recent advances in conservation and population genomics data analysis

“…The reasons for the popularity of GEA analyses are practical: They require no phenotypic data or prior genomic resources, do not require experimental approaches (such as reciprocal transplants) to demonstrate local adaptation, and are often more powerful than differentiation‐based outlier detection methods (De Mita et al., 2013; de Villemereuil, Frichot, Bazin, François, & Gaggiotti, 2014; Forester, Lasky, Wagner, & Urban, 2018; Lotterhos & Whitlock, 2015). In particular, participants considered how and why detection rates differed between univariate and multivariate GEAs, exploring the use of latent factor mixed models (Frichot, Schoville, Bouchard, & Francois, 2013) and redundancy analysis (Forester, Jones, Joost, Landguth, & Lasky, 2016; Lasky et al., 2012), respectively.…”