Quantifying data quality in a citizen science monitoring program: False negatives, false positives and occupancy trends

Abstract

Data collected by volunteers are an important source of information used in species management decisions, yet concerns are often raised over the quality of such data.Two major forms of error exist in occupancy datasets; failing to observe a species when present (imperfect detection-also known as false negatives), and falsely reporting a species as present (false-positive errors). Estimating these rates allows us to quantify volunteer data quality, and may prevent the inference of erroneous trends. We use a new parameterization of a dynamic occupancy model to estimate and adjust for false-negative and false-positive errors, producing accurate estimates of occupancy. We validated this model using simulations and applied it to 12 species datasets collected from a 15-year, large-scale volunteer amphibian monitoring program. False-positive rates were low for most, but not all, species, and accounting for these errors led to quantitative differences in occupancy, although trends remained consistent even when these effects were ignored. We present a model that represents an intuitive way of quantifying the quality of volunteer monitoring datasets, and which can produce unbiased estimates of occupancy despite the presence of multiple types of observation error. Importantly, this allows the quality of volunteer monitoring data to be assessed without relying on comparisons with expert data.

Assessing the usefulness of citizen science data for habitat suitability modelling: Opportunistic reporting versus sampling based on a systematic protocol

“…Also the risk of misidentification varies depending on both rarity and observer skills, with more false‐positive reports of rare species by skilled observers but more false‐positive reports of common species by less experienced observers (Farmer, Leonard, & Horn, 2012). The impact of misidentifications on model performance is difficult to assess and vary between studies (Cruickshank, Bühler, & Schmidt, 2019; Ruiz‐Gutierrez, Hooten, & Campbell Grant, 2016). We assume that the risk of misidentification of rare species is quite low in our dataset due to high self‐validation control carried out by the bird watching community.…”

Occupancy-collection models: Towards bias-corrected modeling of species’ distributions using unstructured occurrence data from museums and herbaria

“…Carefully considering sources of variation in both detection and occupancy probabilities, as well as including random effects can help ensure that all sources of heterogeneity in the data is accounted for. The issue of false identifications has been well-documented in the citizen science literature (Ruiz-Gutierrez et al 2016, Cruickshank et al 2019), which can be addressed in the modeling process (Miller et al 2011). Finally, a key decision of adapting unstructured data for use in an occupancy-detection framework requires careful consideration of how to define spatial units and surveys.…”

Ready-to-use workflows for the implementation of genetic tools in conservation management

“…However, the situation is not really different from field observations of species; an approach to which practitioners are well used. Species can also be misidentified and overlooked in the field, leading to false negatives and false positives in observations as well (Cruickshank et al 2019). In the workflow for the detection of amphibian species with eDNA from water samples by metabarcoding we thus decided to actively communicate uncertainties in species identification, by introducing a sample-specific threshold of the required number of sequencing reads for certain species detection (presence) or uncertain species detection (uncertain detection), respectively (see above).…”