ROBITT: A tool for assessing the risk‐of‐bias in studies of temporal trends in ecology

Abstract

1. Aggregated species occurrence and abundance data from disparate sources are increasingly accessible to ecologists for the analysis of temporal trends in biodiversity. However, sampling biases relevant to any given research question are often poorly explored and infrequently reported; this can undermine statistical inference. In other disciplines, it is common for researchers to complete 'risk-ofbias' assessments to expose and document the potential for biases to undermine conclusions. The huge growth in available data, and recent controversies surrounding their use to infer temporal trends, indicate that similar assessments are urgently needed in ecology.2. We introduce ROBITT, a structured tool for assessing the 'Risk-Of-Bias In studies of Temporal Trends in ecology'. ROBITT has a similar format to its counterparts in other disciplines: it comprises signalling questions designed to elicit information on the potential for bias in key study domains. In answering these, users will define study inferential goal(s) and relevant statistical target populations. This information is used to assess potential sampling biases across domains relevant to the research question (e.g. geography, taxonomy, environment), and how these vary through time. If assessments indicate biases, then users must clearly describe them and/or explain what mitigating action will be taken.3. Everything that users need to complete a ROBITT assessment is provided: the tool, a guidance document and a worked example. Following other disciplines, the tool and guidance document were developed through a consensus-forming process across experts working in relevant areas of ecology and evidence synthesis.

Inferring trends in pollinator distributions across the Neotropics from publicly available data remains challenging despite mobilization efforts

“…It follows that the geographic distribution of sampling has changed over time. This can cause serious problems for the estimation of temporal trends in species' distributions because changes in space are confounded with changes in time (Boyd et al, 2022). For example, a species might fare well in one portion of the continent and less well in another; if the data were sampled from the former portion in one period and the latter portion in the next, then one might come to the artefactual conclusion that the species is in decline.…”

Simple methods for improving the communication of uncertainty in species’ temporal trends

“…For this species, the Frescalo estimates have low uncertainty, and suggest that an increase in the species' 10 km distribution over the last onehundred years is strongly supported. However, the authors of the current paper assessed this conclusion to have a high risk-of-bias (Boyd et al, 2022), due to external knowledge of how this species was treated by plant recorders in the first time period (1930-69;Braithwaite et al, 2006). Risk-of-bias tools typically consist of a set of "domains" against which expert judgement is used to come to some evidence-supported conclusion on the potential for bias within a study.…”

“…An additional species, Potamogeton polygonifolius Pourr., was chosen to demonstrate the fact that model-based uncertainty alone can often be highly misleading, particularly where observational data with potentially serious biases are being used (Boyd et al, 2022;Greenland, 2017). For this species, the Frescalo estimates have low uncertainty, and suggest that an increase in the species' 10 km distribution over the last onehundred years is strongly supported.…”

“…We also demonstrate how broader ontological uncertainty (Spiegelhalter, 2017)-i.e. non-model based uncertainty-can be included in such plots, acknowledging that modelbased uncertainty alone can be very misleading for model/data combinations with a high risk-of-bias (Boyd et al, 2022;Greenland, 2017;van der Bles et al, 2019).…”

An operational workflow for producing periodic estimates of species occupancy at large scales

“…Several tools are emerging to assess biases and other uncertainties in species occurrence data (Boyd et al, 2021;Robin J Boyd et al, 2022b;Zizka et al, 2019Zizka et al, , 2021. One example is the R package occAssess, which takes a dataset and returns several heuristics indicating the potential for spatial, temporal, taxonomic and environmental biases (Boyd et al, 2021;Box 1).…”