Camera settings and biome influence the accuracy of citizen science approaches to camera trap image classification

Egna, Nicole; O'Connor, D.A.; Stacy‐Dawes, Jenna; Tobler, Mathias W.; Pilfold, Nicholas W.; Neilson, Kristin; Simmons, Brooke; Davis, Elizabeth; Bowler, Mark; Fennessy, Julian; Glikman, Jenny Anne; Larpei, Lexson; Lekalgitele, Jesus; Lekupanai, Ruth; Lekushan, Johnson; Lemingani, Lekuran; Lemirgishan, Joseph; Lenaipa, Daniel; Lenyakopiro, Jonathan; Lesipiti, Ranis; Lororua, Masenge; Muneza, Arthur B.; Rabhayo, Sebastian; Ranah, Symon Masiaine Ole; Ruppert, Kirstie; Owen, Megan A.

doi:10.1002/ece3.6722

Cited by 7 publications

(7 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Instead, in the task of detecting presence or absence of iguanas within an image, we found that the answer selected by five volunteers or more (from 20 or 30 total classifications) — which we refer to as the minimum threshold — was the most likely to be correct. This was somewhat unexpected, since previous work has shown that accuracy increased asymptotically with number of classifications per image (Swanson et al, 2016); however, other projects hosted by Zooniverse have found outcomes more similar to ours (Egna et al, 2020; Hennon et al, 2015; Lawson et al, 2022). We suspect that for some challenging images, highly skilled volunteers were required to identify the iguanas, and that these volunteers were relatively rare.…”

Section: Discussionsupporting

confidence: 51%

“…Multiple studies have found that researchers can obtain accurate data from volunteers by properly aggregating multiple independent responses for one subject (e.g. an image or audio file; Anton et al, 2018; Egna et al, 2020; Swanson et al, 2016; Torney et al, 2019). Recent research has identified the number of independent classifications required to give accurate results, thereby streamlining the approach whilst still obtaining scientifically valid outcomes (Swanson et al, 2015).…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Iguanas from above: Citizen scientists provide reliable counts of endangered Galápagos marine iguanas from drone imagery

Varela-Jaramillo,

Winkelmann,

Rivas-Torres

et al. 2024

Preprint

View full text Add to dashboard Cite

Population surveys are vital for wildlife management, yet traditional methods often demand excessive time and resources, leading to data gaps for many species. Modern technologies such as drones can facilitate field surveys but may also increase data analysis challenges. Citizen Science (CS) can address this issue by engaging non-specialists for data collection and analysis. We evaluated CS for population monitoring using the endangered Galapagos marine iguana as a case study, assessing online volunteers' ability to detect and count animals in aerial images. Comparing against a Gold Standard dataset of expert counts in 4345 images, we explored optimal aggregation methods from CS inputs, considering image quality and filtering data from infrequent and anonymous participants. During three phases of our project - hosted on the Zooniverse platform - over 13,000 volunteers made 1,375,201 classifications from 57,838 aerial images; each being independently classified 20 (phases 1 & 2) or 30 (phase 3) times. Volunteers achieved 68% to 94% accuracy in detecting iguanas, with more false negatives than false positives. Image quality strongly influenced accuracy; by excluding data from suboptimal pilot-phase images, volunteers counted with 90% to 92% of accuracy. For detecting presence or absence of iguanas, the commonly used "majority vote" aggregation approach (where the answer selected is that given by the majority of individual inputs) produced less accurate results than when a minimum threshold of five (from the 20/30 independent classifications) was used. For counting iguanas, HDBSCAN clustering yielded the best results. Excluding inputs from anonymous and inexperienced volunteers decreased accuracy. We conclude that online volunteers can accurately identify and count marine iguanas from drone images, though a tendency to underestimate warrants further consideration. CS-based data analysis is faster than manual counting but still resource-intensive, underscoring the need to develop a Machine Learning approach.

show abstract

Section: Discussionsupporting

confidence: 51%

Section: Discussionmentioning

confidence: 99%

Iguanas from above: Citizen scientists provide reliable counts of endangered Galápagos marine iguanas from drone imagery

Varela-Jaramillo,

Winkelmann,

Rivas-Torres

et al. 2024

Preprint

View full text Add to dashboard Cite

show abstract

“…Camera‐trapping citizen science projects have burgeoned recently and been shown to provide ecologically meaningful data (Hsing et al., 2018; Lasky et al., 2021; McShea et al., 2016; Swanson et al., 2016). Factors such as camera settings and location can impact classification accuracy; specifically, sequences with multiple photos have been found to have higher classification accuracy than those with single photos (Egna et al., 2020). Camera trap videos could allow for easier species identification because movement can make animals easier to locate within the footage, and because more information is available to an observer, such as different views of an animal, their gait or movement profile, and sound.…”

Section: Introductionmentioning

confidence: 99%

“…However, probably owing to the concerns outlined above, most camera trap citizen science projects use photographs and there has been little assessment of citizen science classification accuracy of videos (but see McCarthy et al., 2021). Gaining adequate numbers of classifications is important for timely processing and for combining multiple classifications to achieve higher confidence in classification accuracy (Anton et al., 2018; Egna et al., 2020; Hsing et al., 2018; Swanson et al., 2016). Attracting participants and maintaining engagement are, therefore, important considerations for citizen science projects (Meek & Zimmermann, 2016).…”

Section: Introductionmentioning

confidence: 99%

Camera trapping with photos and videos: implications for ecology and citizen science

Green

Stephens

Whittingham

et al. 2022

Remote Sens Ecol Conserv

View full text Add to dashboard Cite

Camera traps are increasingly used in wildlife monitoring and citizen science to address an array of ecological questions on a wide variety of species. However, despite the ability of modern camera traps to capture high-quality video, the majority of studies collect still images, in part because of concerns with video performance. We conducted a camera trap survey of a forested landscape in the UK, using a grid of paired camera traps, to quantify the impact of using video compared to photos on the outcomes of ecological research and for participation and engagement of citizen scientists. Ecological outputs showed no difference between photo and video datasets, but comparison between expert and citizen science classifications showed citizen scientists were able to classify videos more accurately (average accuracy of 95% for video, 86% for photo). Furthermore, citizen scientists were more likely to volunteer additional information on age (provided for 61% videos and 30% photos) and sex (provided for 63% videos and 45% photos) of animals in video footage. Concerns over slow trigger speeds for videos did not appear to affect our datasets or the inferences gained. When combined with citizen science, video datasets are likely to be of higher quality due to increased classification accuracy. Consequently, we encourage researchers to consider the use of video for future camera-trapping projects.

show abstract

“…One potential limitation of this approach is the unknown reliability of the information obtained because of the risk of species misidentification (Miller et al 2012, Egna et al 2020). Some methods can reduce this limitation, such as checking the reliability of responses by asking the interviewees to identify the species in pictures (Wiggins et al 2011, Clare et al 2019, Balázs et al 2021).…”

mentioning

confidence: 99%

Assessing the distribution of elusive non‐game carnivores: are hunters valuable informants?

et al. 2023

Self Cite

View full text Add to dashboard Cite

Public surveys can be valuable tools to collect information on wildlife distribution. Our objective was to improve the knowledge on the distribution and expanding areas of the Egyptian mongoose (Herpestes ichneumon) in central Spain by using an on-line questionnaire to hunters. We assessed the reliability of information that hunters provided by comparing it with reports from wildlife professionals. Our results show a high overall congruence but significantly different information provided by hunters and professionals on the distribution of the Egyptian mongoose, suggesting that both sources provide unique data for the management of non-game species. For example, hunters reported greater mongoose presence than professionals in areas to the east of Toledo and Ciudad Real provinces, for the whole distribution and core area of the species, suggesting that hunters could help to detect other areas where the species exists. Overall, our findings stress the importance of

show abstract

Camera settings and biome influence the accuracy of citizen science approaches to camera trap image classification

Abstract: This is an open access article under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in any medium, provided the original work is properly cited.

Cited by 7 publications

References 42 publications

Iguanas from above: Citizen scientists provide reliable counts of endangered Galápagos marine iguanas from drone imagery

Iguanas from above: Citizen scientists provide reliable counts of endangered Galápagos marine iguanas from drone imagery

Camera trapping with photos and videos: implications for ecology and citizen science

Assessing the distribution of elusive non‐game carnivores: are hunters valuable informants?

Contact Info

Product

Resources

About