Increased adoption of best practices in ecological forecasting enables comparisons of forecastability

Abstract

Near-term iterative forecasting is a powerful tool for ecological decision support and has the potential to transform our understanding of ecological predictability. However, to this point, there has been no cross-ecosystem analysis of near-term ecological forecasts, making it difficult to synthesize diverse research efforts and prioritize future developments for this emerging field. In this study, we analyzed 178 near-term (≤10-year forecast horizon) ecological forecasting papers to understand the development and current state of near-term ecological forecasting literature and compare forecast accuracy across scales and variables. Our results indicate that near-term ecological forecasting is widespread and growing: forecasts have been produced for sites on all seven continents and the rate of forecast publication is increasing over time. As forecast production has accelerated, a number of best practices have been proposed and application of these best practices is increasing. In particular, data publication, forecast archiving, and workflow automation have all increased significantly over time. However, adoption of proposed best practices remains low overall: for example, despite the fact that uncertainty is often cited as an essential component of an ecological forecast, only 45% of papers included uncertainty in their forecast outputs. As the use of these proposed best practices increases, near-term ecological forecasting has the potential to make significant contributions to our understanding of forecastability across scales and variables. In this study, we found that forecastability (defined here as realized forecast accuracy) decreased in predictable patterns over 1–7 day forecast horizons. Variables that were closely related (i.e., chlorophyll and phytoplankton) displayed very similar trends in forecastability, while more distantly related variables (i.e., pollen and evapotranspiration) exhibited significantly different patterns. Increasing use of proposed best practices in ecological forecasting will allow us to examine the forecastability of additional variables and timescales in the future, providing a robust analysis of the fundamental predictability of ecological variables.

PERFICT: A Re‐imagined foundation for predictive ecology

“…This creates robust and nimble models for a range of ecological applications including iterative forecasting cycles (Dietze et al, 2018). Ecologists are embracing modern predictive approaches (Lewis, Woelmer, et al, 2021), benefitting decision‐ and policy‐making for ecosystems worldwide. The PERFICT formalisation can facilitate data‐model integration, and tighten science‐policy integration, because the nimbleness that can come from reusable and interoperable modules for a modelling workflow allows science to respond rapidly to changing policy demands (Table 1; Supporting Information C and D).…”

“…When model workflows are interoperable, generic validation modules can be developed to compare multiple models more quickly (e.g., Barros et al in review), and transferring models to new contexts can help with situations with insufficient data for validation. Validation approaches developed by numerous forecasting efforts (Lewis, Woelmer, et al, 2021) could be more broadly applied using reusable and modular workflow steps.…”

“…For ecologists to improve the quality of predictions, making models that have good assessments of statistical fit is insufficient; we must make and learn from many predictions by comparing to out-of-sample data (Lewis, Woelmer, et al, 2021;Tetlock & Gardner, 2016). Forecasting challenges are forcing some ecologists to do this (e.g., the Ecological Forecasting Challenge with NEON; https://ecofo recast.org/efi-rcn-forec ast-chall enges/).…”

“…The current biodiversity crisis and increasing pressures on socio‐ecological systems (e.g., climate, land use, and pollutants) present time‐sensitive challenges to ecosystem and landscape management, and sustainable development (McPhearson et al, 2021). As a result, the number of applied ecological models has exploded in recent decades (e.g., repositories for NetLogo Wilensky, 1999; EwE Christensen & Walters, 2004) and calls for iterative forecasting have been widely embraced (Lewis, Woelmer, et al, 2021). A wider application of these models would likely help to solve these challenges, as this would facilitate comparing models, building model ensembles, and testing hypotheses (Belete et al, 2017; Wenger & Olden, 2012).…”

“…Predictive Ecology, a branch of ecology based on quantitative deductions from models (Houlahan et al, 2015;McGill et al, 2007;Mouquet et al, 2015;Peters, 1977Peters, , 1991Travers et al, 2019), provides a framing that helps with these challenges. However, it falls short in its current form; it is focused on models per se, not the workflow involved in generating predictions (Fer et al, 2021;Lewis, Woelmer, et al, 2021). Prediction is important in model evaluation (e.g., cross-validation) and is important in forecasting (i.e., predictions of future conditions).…”