Spatial leave‐one‐out cross‐validation for variable selection in the presence of spatial autocorrelation

Abstract

Aim Processes and variables measured in ecology are almost always spatially autocorrelated, potentially leading to the choice of overly complex models when performing variable selection. One way to solve this problem is to account for residual spatial autocorrelation (RSA) for each subset of variables considered and then use a classical model selection criterion such as the Akaike information criterion (AIC). However, this method can be laborious and it raises other concerns such as which spatial model to use or how to compare different spatial models. To improve the accuracy of variable selection in ecology, this study evaluates an alternative method based on a spatial cross-validation procedure. Such a procedure is usually used for model evaluation but can also provide interesting outcomes for variable selection in the presence of spatial autocorrelation.Innovation We propose to use a special case of spatial cross-validation, spatial leave-one-out (SLOO), giving a criterion equivalent to the AIC in the absence of spatial autocorrelation. SLOO only computes non-spatial models and uses a threshold distance (equal to the range of RSA) to keep each point left out spatially independent from the others. We first provide some simulations to evaluate how SLOO performs compared with AIC. We then assess the robustness of SLOO on a large-scale dataset. R software codes are provided for generalized linear models. Main conclusionsThe AIC was relevant for variable selection in the presence of RSA if the independent variables considered were not spatially autocorrelated. It otherwise failed because highly spatially autocorrelated variables were more often selected than others. Conversely, SLOO had similar performances whether the variables were themselves spatially autocorrelated or not. It was particularly useful when the range of RSA was small, which is a common property of spatial tools. SLOO appears to be a promising solution for selecting relevant variables from most ecological spatial datasets.

Cross‐validation strategies for data with temporal, spatial, hierarchical, or phylogenetic structure

“…Early solutions were leave‐ n ‐out cross‐validation approaches (Stone , Picard and Cook ) that run iteratively, each time withholding a small randomly selected subset of the data for testing. Because these approaches have also been shown to produce biased error estimates (Shao , Kohavi , Telford et al , Amesbury et al ), further corrections have been proposed, for example by incorporating distance‐based buffers around hold‐out points (Bahn , Telford and Birks , Le Rest et al ).…”

Caveats for correlative species distribution modeling

“…Spatial autocorrelation in residuals can be addressed by either including spatial structure in the model error term (e.g., Lichstein et al, 2002) or using additional predictor variable(s) that are ecologically meaningful and also match the spatial pattern of the residuals. The spatial leave-oneout procedure suggested by Le Rest et al (2014) , which is a special case of spatial cross-validation where each point left out is spatially independent from the others, can also be used to account for residual spatial autocorrelation. Higher spatial autocorrelation in species occurrence data can result in highly inflated test evaluation statistics and erroneous model projections (Boria et al, 2014).…”

Evaluation of diagnostics for hierarchical spatial statistical models

“…When a diagnostic is evaluated using the same data that were used to fit the model, the results are well known to be biased (Bayarri and Berger 2000;Dahl 2006;Efron 1986;Hjort et al 2006). An alternative is to use cross-validation (Gelfand 1996;Le Rest et al 2014;Stone 1974;Zhang and Wang 2010), where the model is fitted to m < n observations and evaluated using the remaining n − m observations. While crossvalidation is considered a gold standard for diagnostics (Gelfand et al 1992;Marshall and Spiegelhalter 2003;Stern and Cressie 2000), it is computationally expensive and may be impractical for very large datasets.…”