Digital taxonomist: Identifying plant species in community scientists’ photographs

Lutio, Riccardo de; She, Yihang; D’Aronco, Stefano; Russo, Stefania; Brun, Philipp; Wegner, Jan Dirk; Schindler, Konrad

doi:10.1016/j.isprsjprs.2021.10.002

Cited by 14 publications

(21 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this study, we present a novel method that fuses image and temporal metadata for recognizing wildlife in camera trap images. Our experimental results on different camera trap datasets demonstrate that leveraging temporal metadata can improve overall wildlife recognition performance, which is similar to the findings of Terry et al [15] and de Lutio et al [16] who utilized contextual data to enhance the recognition performance of citizen science images.…”

Section: Discussionsupporting

confidence: 87%

“…Since dates have a cyclical feature, the end of one year and the start of the next year should be close to each other. Therefore, we use sine-cosine mapping [16] to encode the date metadata captured by the camera trap as (d 1 , d 2 ) according to Equations ( 5) and (6). With this cyclical encoding, December 31st and January 1st are mapped to be near each other.…”

Section: Temporal Feature Extractionmentioning

confidence: 99%

“…Terry et al [15] developed a multi-input neural network model that fuses contextual metadata and images to identify ladybird species in the British Isles, UK, demonstrating that deep learning models can effectively use contextual information to improve the top-1 accuracy of multi-input models from 48.2% to 57.3%. de Lutio et al [16] utilized the spatial, temporal, and ecological contexts attached to most plant species' observation information to construct a digital taxonomist that improved accuracy from 73.48% to 79.12% compared to a model trained using only images. Mou et al [17] used animals' visual features, for example, the color of a bird's feathers or the color of an animal's fur, to improve the recognition accuracy of a contrastive language-image pre-trained (CLIP) model on multiple animal datasets.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Improved Wildlife Recognition through Fusing Camera Trap Images and Temporal Metadata

Liu,

Mou,

2024

Diversity

View full text Add to dashboard Cite

Camera traps play an important role in biodiversity monitoring. An increasing number of studies have been conducted to automatically recognize wildlife in camera trap images through deep learning. However, wildlife recognition by camera trap images alone is often limited by the size and quality of the dataset. To address the above issues, we propose the Temporal-SE-ResNet50 network, which aims to improve wildlife recognition accuracy by exploiting the temporal information attached to camera trap images. First, we constructed the SE-ResNet50 network to extract image features. Second, we obtained temporal metadata from camera trap images, and after cyclical encoding, we used a residual multilayer perceptron (MLP) network to obtain temporal features. Finally, the image features and temporal features were fused in wildlife identification by a dynamic MLP module. The experimental results on the Camdeboo dataset show that the accuracy of wildlife recognition after fusing the image and temporal information is about 93.10%, which is an improvement of 0.53%, 0.94%, 1.35%, 2.93%, and 5.98%, respectively, compared with the ResNet50, VGG19, ShuffleNetV2-2.0x, MobileNetV3-L, and ConvNeXt-B models. Furthermore, we demonstrate the effectiveness of the proposed method on different national park camera trap datasets. Our method provides a new idea for fusing animal domain knowledge to further improve the accuracy of wildlife recognition, which can better serve wildlife conservation and ecological research.

show abstract

Section: Discussionsupporting

confidence: 87%

Section: Temporal Feature Extractionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Improved Wildlife Recognition through Fusing Camera Trap Images and Temporal Metadata

Liu,

Mou,

2024

Diversity

View full text Add to dashboard Cite

show abstract

“…The use of measurements taken from photographs allowed us to provide a robust approach in species' assignment, as already common for individual‐based recognition methods (De Lutio et al., 2021; Ferreira et al., 2020), a factor that is key to the use of citizen‐science records, particularly in the case of cryptic species (Gorleri et al., 2022). We acknowledge that our approach is only applicable to cryptic species with identifiable morphological differentiation that correspond to levels 2 and 3 of the classification proposed by Chenuil et al., 2019.…”

Section: Discussionmentioning

confidence: 99%

Hidden in plain sight: unveiling the distributions of green‐winged grasshoppers (Aiolopus spp.) with citizen‐science data

Pélissié¹,

Labadessa

Ancillotto

2023

Journal of Zoology

View full text Add to dashboard Cite

Understanding the geographical distribution of phenotypically highly similar species (i.e. cryptic species) represents a challenge to biogeographers, due to the obvious difficulties in identifying such taxa without specific expertise. Besides, citizen science is increasingly emerging as a key approach for supporting biodiversity data collection, but remains hard to apply in the case of cryptic species. Here we aim to test the combination of community records and photography‐based quantitative methods, for assessing the distribution of cryptic taxa, by using two grasshopper species of the genus Aiolopus as models. To achieve these objectives, we first assess the reliability of photography‐based diagnostic criteria to differentiate between A. thalassinus and A. puissanti without ambiguity from correctly identified records, and then apply such criteria to geographical regions of potential range overlap between the two species, in order to clarify their respective distributions. By applying a multivariate classification approach based on ratio measurements taken from photographs, we provide a quantitative tool to successfully identify the two species, and disclose that A. puissanti widely occurs outside of its currently known range, and outline potential research avenues on the biogeography of these poorly studied species. Our results also point at how some types of cryptic species may be effectively identified by adopting a quantitative photography‐based approach, with applicability for clarifying species' distributions at wide scales by exploiting publicly available citizen‐science records. Our study thus, besides shedding light onto the biogeography and distributions of Aiolopus grasshoppers across the Mediterranean, represents an effective and repeatable framework to disentangle the distributions of poorly studied cryptic species.

show abstract

“…Deep learning classification is not limited to considering one source of data such as image pixels. Combining images with contextual information about location, date, weather, habitat and user expertise has been shown to improve accuracy (de Lutio et al, 2021; Lin et al, 2021; Picek et al, 2022; Sun, Futahashi, & Yamanaka, 2021; Terry et al, 2020). Other innovations include incorporating taxonomy into the network training process (Elhamod et al, 2021; Zhang et al, 2021) and combining images with DNA sequence data (Wang, Chen, et al, 2021; Yang et al, 2022).…”

Section: Applications In Ecology and Evolutionmentioning

confidence: 99%

Deep learning as a tool for ecology and evolution

et al. 2022

View full text Add to dashboard Cite

Deep learning is driving recent advances behind many everyday technologies, including speech and image recognition, natural language processing and autonomous driving. It is also gaining popularity in biology, where it has been used for automated species identification, environmental monitoring, ecological modelling, behavioural studies, DNA sequencing and population genetics and phylogenetics, among other applications. Deep learning relies on artificial neural networks for predictive modelling and excels at recognizing complex patterns. In this review we synthesize 818 studies using deep learning in the context of ecology and evolution to give a discipline‐wide perspective necessary to promote a rethinking of inference approaches in the field. We provide an introduction to machine learning and contrast it with mechanistic inference, followed by a gentle primer on deep learning. We review the applications of deep learning in ecology and evolution and discuss its limitations and efforts to overcome them. We also provide a practical primer for biologists interested in including deep learning in their toolkit and identify its possible future applications. We find that deep learning is being rapidly adopted in ecology and evolution, with 589 studies (64%) published since the beginning of 2019. Most use convolutional neural networks (496 studies) and supervised learning for image identification but also for tasks using molecular data, sounds, environmental data or video as input. More sophisticated uses of deep learning in biology are also beginning to appear. Operating within the machine learning paradigm, deep learning can be viewed as an alternative to mechanistic modelling. It has desirable properties of good performance and scaling with increasing complexity, while posing unique challenges such as sensitivity to bias in input data. We expect that rapid adoption of deep learning in ecology and evolution will continue, especially in automation of biodiversity monitoring and discovery and inference from genetic data. Increased use of unsupervised learning for discovery and visualization of clusters and gaps, simplification of multi‐step analysis pipelines, and integration of machine learning into graduate and postgraduate training are all likely in the near future.

show abstract

Digital taxonomist: Identifying plant species in community scientists’ photographs

Cited by 14 publications

References 17 publications

Improved Wildlife Recognition through Fusing Camera Trap Images and Temporal Metadata

Improved Wildlife Recognition through Fusing Camera Trap Images and Temporal Metadata

Hidden in plain sight: unveiling the distributions of green‐winged grasshoppers (Aiolopus spp.) with citizen‐science data

Deep learning as a tool for ecology and evolution

Contact Info

Product

Resources

About