The effect of sample size on the accuracy of species distribution models: considering both presences and pseudo‐absences or background sites

Abstract

Most high-performing species distribution modelling techniques require both presences, and either absences or pseudo-absences or background points. In this paper, we explore the effect of sample size, towards developing improved strategies for modelling. We generated 1800 virtual species with three levels of prevalence using ten modelling techniques, while varying the number of training presences (NTP) and the number of random points (NRP representing pseudo-absences or background sites). For five of the ten modelling techniques we built two versions of models: one with an equal total weight (ETW) setting where the total weight for pseudo-absence is equivalent to the total weight for presence, and another with an unequal total weight (UTW) setting where the total weight for pseudo-absence is not required to be equal to the total weight for presence. We compared two strategies for NRP: a small multiplier strategy (i.e. setting NRP at a few times as large as NTP), and a large number strategy (i.e. using numerous random points). We produced ensemble models (by averaging the predictions from 30 models built with the same set of training presences and different sets of random points in equivalent numbers) for three NTP magnitudes and two NRP strategies. We found that model accuracy altered as NRP increased with four distinct patterns of performance: increasing, decreasing, arch-shaped and horizontal. In most cases ETW improved model performance. Ensemble models had higher accuracy than the corresponding single models, and this improvement was pronounced when NTP was low. We conclude that a large NRP is not always an appropriate strategy. The best choice for NRP will depend on the modelling techniques used, species prevalence and NTP. We recommend building ensemble models instead of single models, using the small multiplier strategy for NRP with ETW, especially when only a small number of species presence records are available.

Testing methods in species distribution modelling using virtual species: what have we learnt and what are we missing?

“…Liu et al (2019) argued that, as VS represent simulations, standard statistical analyses cannot be used to analyze the results. Liu et al (2019) argued that, as VS represent simulations, standard statistical analyses cannot be used to analyze the results.…”

“…A fourth question that we see occasionally addressed is what kind of statistical analyses should be applied to VS studies. Liu et al (2019) argued that, as VS represent simulations, standard statistical analyses cannot be used to analyze the results. This was based on an argument advanced by White et al (2014) and dealing with simulated data at large.…”

Effects of climatically shifting species distributions on biocultural relationships

“…Thinned occurrence records were projected onto a 30 arc‐second 2 (~1 km 2 ) gridded map matching the predictor variable data for both Kūmarahou (105 grid cells) and Kuta (941 grid cells). To optimize SDMs, Kūmarahou and Kuta pseudo‐absences (210 and 1,882, respectively; Figures S2 and S3) were selected for a ratio of pseudo‐absence to presence of 2:1 (Barbet‐Massin et al., ; Liu, Newell, & White, ).…”

“…The recent potential distribution for each species was determined from the relationship between the gridded presence/pseudo‐absence data and environmental variables, including climate under recent (1961–1990) conditions (Anderson et al., ). We used seven standard SDMs available in the BIOMOD2 package (Thuiller, Georges, & Engler, ) to increase accuracy and ensure that our results were robust to differences in model type (Elith et al, ; Liu, Newell, & White, ): Artificial Neural Network (ANN), Flexible Discriminant Analysis (FDA), Generalized Boosting Models (GBM), Generalized Additive Model (GAM), Generalized Linear Model (GLM), Multiple Adaptive Regression Splines (MARS), and Random Forest (RF). For each species, SDMs were run in 10 replicates; each replicate was built using 70% of the presence and pseudo‐absence data (randomly sampled) and evaluated with remaining 30% of presence and pseudo‐absence data (Guisan et al., ).…”

“…The resulting thinned data samples retained 105 and 941 points for Kūmarahou and Kuta respectively ( Figures S2 and S3). Thinned occurrence records were projected onto a 30 arc-second 2 (~1 km 2 ) gridded map matching the predictor variable data for both Kūmarahou (105 grid cells) and Kuta We used seven standard SDMs available in the BIOMOD2 package (Thuiller, Georges, & Engler, 2013) to increase accuracy and ensure that our results were robust to differences in model type (Elith et al, 2006;Liu, Newell, & White, 2018) replicates; each replicate was built using 70% of the presence and pseudo-absence data (randomly sampled) and evaluated with remaining 30% of presence and pseudo-absence data (Guisan et al, 2017).…”