Automated visitor and wildlife monitoring with camera traps and machine learning

Abstract

As human activities in natural areas increase, understanding human–wildlife interactions is crucial. Big data approaches, like large‐scale camera trap studies, are becoming more relevant for studying these interactions. In addition, open‐source object detection models are rapidly improving and have great potential to enhance the image processing of camera trap data from human and wildlife activities. In this study, we evaluate the performance of the open‐source object detection model MegaDetector in cross‐regional monitoring using camera traps. The performance at detecting and counting humans, animals and vehicles is evaluated by comparing the detection results with manual classifications of more than 300 000 camera trap images from three study regions. Moreover, we investigate structural patterns of misclassification and evaluate the results of the detection model for typical temporal analyses conducted in ecological research. Overall, the accuracy of the detection model was very high with 96.0% accuracy for animals, 93.8% for persons and 99.3% for vehicles. Results reveal systematic patterns in misclassifications that can be automatically identified and removed. In addition, we show that the detection model can be readily used to count people and animals on images with underestimating persons by −0.05, vehicles by −0.01 and animals by −0.01 counts per image. Most importantly, the temporal pattern in a long‐term time series of manually classified human and wildlife activities was highly correlated with classification results of the detection model (Pearson's r = 0.996, p < 0.001) and diurnal kernel densities of activities were almost equivalent for manual and automated classification. The results thus prove the overall applicability of the detection model in the image classification process of cross‐regional camera trap studies without further manual intervention. Besides the great acceleration in processing speed, the model is also suitable for long‐term monitoring and allows reproducibility in scientific studies while complying with privacy regulations.

Being confident in confidence scores: calibration in deep learning models for camera trap image sequences

“…Secondly, we argue that the integration of machine learning predictions directly into subsequent ecological tasks can be facilitated by achieving calibration. Many ecological downstream tasks (e.g estimating occupancy, abundance or activity patterns) based on deep learning predictions use an arbitrary threshold selection (Lonsinger et al 2023; Krivek et al 2023) to consider that a prediction is correct, or test a series of thresholds to determine the optimal one given known species labels (Whytock, SŚwieżewski, et al 2021; Mitterwallner et al 2023). However, the ultimate goal of using AI is to avoid having to label images.…”

“…In this paper, we explore the calibration of confidence scores in the context of species classification models for camera trap data. For that task, the recurring leading approach, as assessed in recent iWildcam competitions (Beery, Agarwal, et al 2021), consists in two steps: (step 1) detecting animals, humans and vehicles and filtering out empty im-ages using a robust detection model such as MegaDetector (Beery, Morris, et al 2019;Mitterwallner et al 2023) and (step 2) using a convolutional neural network (CNN) classification model to identify the species in the bounding box returned by the detection model, when an animal has been detected. We therefore focus on these species classification models (step 2), which are developed for a large range of species all over the world.…”

Action Detection for Wildlife Monitoring with Camera Traps Based on Segmentation with Filtering of Tracklets (SWIFT) and Mask-Guided Action Recognition (MAROON)

“…Over the last few decades, camera traps have been adopted for various ecological tasks, including abundance estimation [5][6][7], the quantification of species diversity [8], the detection of rare species [9], the investigation of animal activity patterns [10], and the analysis of species replacement processes [11]. Automatic analysis using artificial intelligence is absolutely necessary to deal with the vast amount of collected camera trap data [12,13]. Currently, many AI models are created that enable the management and processing of camera trap images and videos, facilitate the categorization of camera trap images, or are able to classify species within these images [14].…”

Zero-shot animal behavior classification with vision-language foundation models

“…Many machine learning models are now able to reliably predict the species identity of animals seen in camera trap images [12, 17, 25], an information that can be used in several downstream tasks. For example Whytock et al .…”

“…Deriving activity patterns from camera-traps images first requires identifying the species present in the pictures (Whytock et al 2021). This task can now often be fully automatized, as recent machine learning models achieve very high performance in species classification (Rigoudy et al 2023;Willi et al 2019;Mitterwallner et al 2023). Then, activity level is generally assumed to be proportional to the number of sightings.…”