Can Observation Skills of Citizen Scientists Be Estimated Using Species Accumulation Curves?

Abstract

Volunteers are increasingly being recruited into citizen science projects to collect observations for scientific studies. An additional goal of these projects is to engage and educate these volunteers. Thus, there are few barriers to participation resulting in volunteer observers with varying ability to complete the project’s tasks. To improve the quality of a citizen science project’s outcomes it would be useful to account for inter-observer variation, and to assess the rarely tested presumption that participating in a citizen science projects results in volunteers becoming better observers. Here we present a method for indexing observer variability based on the data routinely submitted by observers participating in the citizen science project eBird, a broad-scale monitoring project in which observers collect and submit lists of the bird species observed while birding. Our method for indexing observer variability uses species accumulation curves, lines that describe how the total number of species reported increase with increasing time spent in collecting observations. We find that differences in species accumulation curves among observers equates to higher rates of species accumulation, particularly for harder-to-identify species, and reveals increased species accumulation rates with continued participation. We suggest that these properties of our analysis provide a measure of observer skill, and that the potential to derive post-hoc data-derived measurements of participant ability should be more widely explored by analysts of data from citizen science projects. We see the potential for inferential results from analyses of citizen science data to be improved by accounting for observer skill.

Completeness of citizen science biodiversity data from a volunteered geographic information perspective

“…Colwell, Mao, and Chang 2004). A recent study used SACs of individual eBird observers to estimate their skill (Kelling et al 2015). SACs plot the number of observed species in an area over some measure of effort (e.g.…”

Estimates of observer expertise improve species distributions from citizen science data

“…The expertise score that we examined in this paper was calculated based on information about observed species richness (Kelling et al., ). Here, we established that this score also provides useful information about inter‐observer variation in reporting rates of individual species.…”

“…The model also contained two observer‐specific random effects: the intercept and the slope of the duration of the observation period. The estimated expertise score for each observer was defined as the natural log of the expected number of species that an observer would record during a standardised search in an average habitat (Kelling et al., ).…”

“…Dealing with variation in observer skill at the stage of data analysis involves estimating how reporting rates vary among observers, in order to improve the accuracy and precision of ecological models (Farmer, Leonard, & Horn, ; Isaac et al., ; Tweddle et al., ). Expert observers will be able to identify species from more distant sightings or shorter bursts of song, which will give them higher reporting rates and on average they will record more species, all other things being equal (Kelling et al., ; Pescott et al., ). Moreover, the impact of the heterogeneity among observers is likely to vary across species, potentially biasing analyses that compare species.…”

“…We examine the expertise score described in Kelling et al. (), which was originally developed to examine whether participants in a citizen science project learned by participation (i.e. measurable improvements in detecting and identifying birds).…”