Performance of partial statistics in individual‐based landscape genetics

Abstract

Individual-based landscape genetic methods have become increasingly popular for quantifying fine-scale landscape influences on gene flow. One complication for individual-based methods is that gene flow and landscape variables are often correlated with geography. Partial statistics, particularly Mantel tests, are often employed to control for these inherent correlations by removing the effects of geography while simultaneously correlating measures of genetic differentiation and landscape variables of interest. Concerns about the reliability of Mantel tests prompted this study, in which we use simulated landscapes to evaluate the performance of partial Mantel tests and two ordination methods, distance-based redundancy analysis (dbRDA) and redundancy analysis (RDA), for detecting isolation by distance (IBD) and isolation by landscape resistance (IBR). Specifically, we described the effects of suitable habitat amount, fragmentation and resistance strength on metrics of accuracy (frequency of correct results, type I/II errors and strength of IBR according to underlying landscape and resistance strength) for each test using realistic individual-based gene flow simulations. Mantel tests were very effective for detecting IBD, but exhibited higher error rates when detecting IBR. Ordination methods were overall more accurate in detecting IBR, but had high type I errors compared to partial Mantel tests. Thus, no one test outperformed another completely. A combination of statistical tests, for example partial Mantel tests to detect IBD paired with appropriate ordination techniques for IBR detection, provides the best characterization of fine-scale landscape genetic structure. Realistic simulations of empirical data sets will further increase power to distinguish among putative mechanisms of differentiation.

Fine-scale landscape genetics of the American badger (Taxidea taxus): disentangling landscape effects and sampling artifacts in a poorly understood species

“…Our study demonstrates that combining ordination and regression statistics with simulations to quantify errors can help disentangle potential errors from landscape effects on gene flow in an individual-based framework. RDA and spatially lagged regression did not require calculation of connectivity indices (Kierepka and Latch, 2015), which was critical for this study given the nearly complete lack of relevant life history data. With simulations, we were able to quantify the confounding effects of spatially biased sampling to separate type I errors from biologically relevant landscape effects on gene flow.…”

“…We performed a partial RDA, a constrained ordination technique that is the multivariate analog to simple linear regression (Legendre and Legendre, 1998). Previous simulation studies in both population- (Fortín and Legendre, 2010) and individual-based (Kierepka and Latch, 2015) studies have demonstrated that RDA has much greater power than Mantel tests to detect relationships in autocorrelated data. In individual-based studies, however, RDA can suffer from type I errors (that is, false significance; Kierepka and Latch, 2015), so additional steps are needed to control for any potential errors.…”

“…Previous simulation studies in both population- (Fortín and Legendre, 2010) and individual-based (Kierepka and Latch, 2015) studies have demonstrated that RDA has much greater power than Mantel tests to detect relationships in autocorrelated data. In individual-based studies, however, RDA can suffer from type I errors (that is, false significance; Kierepka and Latch, 2015), so additional steps are needed to control for any potential errors. The spatially lagged scores from the two retained sPCA axes were analyzed together as explanatory variables with both latitude and longitude in the conditional matrix (that is, variables to be controlled for in final models).…”

“…First, these methods offer considerable flexibility in the type of landscape and genetic variables that can be evaluated, negating reliance on a priori parameterization of landscape resistance hypotheses. Furthermore, ordination and regression techniques are considered more robust than correlation statistics (for example, Mantel tests; Fortín and Legendre, 2010), and therefore, may limit type I or II errors (Balkenhol et al, 2009;Kierepka and Latch, 2015). Recent studies have demonstrated the utility of ordination and regression techniques for detecting complex, interacting influences on gene flow within continuously distributed, well-studied species but have applied these techniques in a population-based framework (that is, between genetic clusters, Reding et al, 2012; or between sampled areas, Blanchong et al, 2008;Robinson et al, 2012).…”

A spatial genetics approach to inform vector control of tsetse flies (Glossina fuscipes fuscipes) in Northern Uganda

“…Mantel tests remain common in landscape genetics (Manel et al., 2003) and can perform better than other methods when the assumption of linearity is not violated (Kierepka & Latch, 2015; Shirk, Landguth, & Cushman, 2017; Zeller et al., 2016). However, use of Mantel tests is controversial because of weakness in accounting for spatial autocorrelation (Legendre & Fortin, 2010; Manel et al., 2003), so we interpret results of the partial Mantel test with caution and as a supplement to the results of the MMRR.…”