Model selection with multiple regression on distance matrices leads to incorrect inferences

Abstract

In landscape genetics, model selection procedures based on Information Theoretic and Bayesian principles have been used with multiple regression on distance matrices (MRM) to test the relationship between multiple vectors of pairwise genetic, geographic, and environmental distance. Using Monte Carlo simulations, we examined the ability of model selection criteria based on Akaike’s information criterion (AIC), its small-sample correction (AICc), and the Bayesian information criterion (BIC) to reliably rank candidate models when applied with MRM while varying the sample size. The results showed a serious problem: all three criteria exhibit a systematic bias toward selecting unnecessarily complex models containing spurious random variables and erroneously suggest a high level of support for the incorrectly ranked best model. These problems effectively increased with increasing sample size. The failure of AIC, AICc, and BIC was likely driven by the inflated sample size and different sum-of-squares partitioned by MRM, and the resulting effect on delta values. Based on these findings, we strongly discourage the continued application of AIC, AICc, and BIC for model selection with MRM.

Demographic and genetic approaches to study dispersal in wild animal populations: A methodological review

“…Linked‐based causal models (e.g., Cushman, McKelvey, Hayden, & Schwartz, ; Fourtune et al., ; Wang et al., ) are a promising alternative to the previously described models as they allow inferring causal relationships among the genetic response and landscape predictors beyond simple correlations, although they may also be sensitive to collinearity. Finally, most linked‐based statistical models may be subject to model selection procedures based on the comparison of model fit parameters (e.g., Keller, Holderegger, & van Strien, ) or Akaike information criterion (Burnham & Anderson, ; but see Prunier et al., ; Franckowiak et al., ).…”

Comparative landscape genetics of two endemic torrent salamander species, Rhyacotriton kezeri and R. variegatus: implications for forest management and species conservation

“…Canopy cover, frostfree period, and growing season precipitation were downloaded in the form of raster files. Compound topographic index, heat load index, roughness, and slope were calculated from the National Elevation Dataset digital elevation model (Gesch et al 2002;Gesch 2007) using the Geomorphometry and Gradient Metrics toolbox v2.0-0 for ArcGIS (Evans et al 2014). Compound topographic index is an estimate of soil wetness based on the expected movement of water downhill, heat load index reflects the amount of solar radiation reaching a surface, and roughness quantifies the change in elevation per unit area (Evans et al 2014).…”

“…Compound topographic index, heat load index, roughness, and slope were calculated from the National Elevation Dataset digital elevation model (Gesch et al 2002;Gesch 2007) using the Geomorphometry and Gradient Metrics toolbox v2.0-0 for ArcGIS (Evans et al 2014). Compound topographic index is an estimate of soil wetness based on the expected movement of water downhill, heat load index reflects the amount of solar radiation reaching a surface, and roughness quantifies the change in elevation per unit area (Evans et al 2014). To test for nonlinear relationships between these continuous variables and genetic distance, after rescaling all variables to range from 0 to 1 we created two transformed resistance surfaces for each variable: T1 = 100 (originalcostsurface) , and T2 = 100 − 100 (1−originalcostsurface) .…”

Landscape does matter: Disentangling founder effects from natural and human‐aided post‐introduction dispersal during an ongoing biological invasion

“…If the probability associated with X 2 and X 3 variables became non‐significant, we selected two‐factor models as the best models. Based on previous examination of possible biases introduced by classical model selection methods used in combination with MRM (Franckowiak et al., 2017), the best model was thus selected as the one explaining the largest proportion of genetic variation (i.e. with the best R 2 ).…”