Automatic flower detection and phenology monitoring using time‐lapse cameras and deep learning

Abstract

The advancement of spring is a widespread biological response to climate change observed across taxa and biomes. However, the species level responses to warming are complex and the underlying mechanisms are difficult to disentangle. This is partly due to a lack of data, which are typically collected by direct observations, and thus very time-consuming to obtain. Data deficiency is especially pronounced in the Arctic where the warming is particularly severe. We present a method for automated monitoring of flowering phenology of specific plant species at very high temporal resolution through full growing seasons and across geographical regions. The method consists of image-based monitoring of field plots using near-surface time-lapse cameras and subsequent automated detection and counting of flowers in the images using a convolutional neural network. We demonstrate the feasibility of collecting flower phenology data using automatic time-lapse cameras and show that the temporal resolution of the results surpasses what can be collected by traditional observation methods. We focus on two Arctic species, the mountain avens Dryas octopetala and Dryas integrifolia in 20 image series from four sites. Our flower detection model proved capable of detecting flowers of the two species with a remarkable precision of 0.918 (adjusted to 0.966) and a recall of 0.907. Thus, the method can automatically quantify the seasonal dynamics of flower abundance at fine scale and return reliable estimates of traditional phenological variables such as the timing of onset, peak, and end of flowering. We describe the system and compare manual and automatic extraction of flowering phenology data from the images. Our method can be directly applied on sites containing mountain avens using our trained model, or the model could be fine-tuned to other species. We discuss the potential of automatic image-based monitoring of flower phenology and how the method can be improved and expanded for future studies.

Moths complement bumblebee pollination of red clover: a case for day-and-night insect surveillance

“…Ultimately, cameras should play a central role in future monitoring schemes for plant–pollinator interactions. Standardized image libraries can provide a permanent archive of plant and insect phenology, and train deep-learning models to automatically extract ecological information [ 26 , 35 , 36 ]. Using both cameras and established methods, science will converge on the true extent of pollinator declines—but also the most appropriate remedial interventions.…”

Towards the fully automated monitoring of ecological communities

“…Using on‐board data processing approaches can minimise storage to only critical and informative components, extending battery life and storage capacity (Liu et al, 2019 ). For example, time‐lapse wildlife camera systems powered by lithium AA batteries can run remotely for several months without human intervention, except for replacing SD cards (Mann et al, 2022 ). For systems with internet access, the introduction of 5G cellular networks and specialised networks for the Internet of Things (e.g.…”

“…For example, a majority vote can be taken across consecutive classifications of an individual insect. A tracking algorithm can also be deployed to identify and separate individual flowers; this allows derivation of flower‐level data on floral traits, phenology and visitation (Mann et al, 2022 ). Such real‐time detection and classification present exciting opportunities to examine species interactions at unprecedented spatiotemporal resolutions.…”

“…For example, a majority vote can be taken across consecutive classifications of an individual insect. A tracking algorithm can also be deployed to identify and separate individual flowers; this allows derivation of flower‐level data on floral traits, phenology and visitation (Mann et al, 2022 ).…”

Accurate detection and identification of insects from camera trap images with deep learning

“…With our study, we provide the tools and benchmark data to advance the implementation of robust automated monitoring of insects in situ. Our system is widely applicable as a means to capture images of plants and insects and to automatically generate monitoring data of insect species abundance (4,(22)(23)(24). Our method could also contribute to insect pest monitoring with camera-equipped traps in agriculture and forestry without killing rare insect species (25,26).…”