Automatic Fungi Recognition: Deep Learning Meets Mycology

Abstract

The article presents an AI-based fungi species recognition system for a citizen-science community. The system’s real-time identification too — FungiVision — with a mobile application front-end, led to increased public interest in fungi, quadrupling the number of citizens collecting data. FungiVision, deployed with a human-in-the-loop, reaches nearly 93% accuracy. Using the collected data, we developed a novel fine-grained classification dataset — Danish Fungi 2020 (DF20) — with several unique characteristics: species-level labels, a small number of errors, and rich observation metadata. The dataset enables the testing of the ability to improve classification using metadata, e.g., time, location, habitat and substrate, facilitates classifier calibration testing and finally allows the study of the impact of the device settings on the classification performance. The continual flow of labelled data supports improvements of the online recognition system. Finally, we present a novel method for the fungi recognition service, based on a Vision Transformer architecture. Trained on DF20 and exploiting available metadata, it achieves a recognition error that is 46.75% lower than the current system. By providing a stream of labeled data in one direction, and an accuracy increase in the other, the collaboration creates a virtuous cycle helping both communities.

Deep learning as a tool for ecology and evolution

“…Finally, network architectures are constantly evolving and improving accuracy on benchmark data. The recently introduced transformer architectures have already replaced RNNs in many language processing tasks (Edwards et al, 2022; Vaswani et al, 2017) and their variants may even supersede CNNs in image recognition (Dosovitskiy et al, 2020; Liu et al, 2021; Picek et al, 2022).…”

“…Given its utility for automated identification, deep learning is increasingly used in community science initiatives. Examples include a growing number of mobile phone applications such bird identification tool Merlin or the citizen naturalist portal iNaturalist, as well as a number of more local or taxon‐specific guides (Farnsworth et al, 2013; Kahl et al, 2021; Picek et al, 2022; Sulc et al, 2020; Wäldchen & Mäder, 2018). Many of these applications crowd‐source training data collection and identification verification by users.…”

“…Deep learning classification is not limited to considering one source of data such as image pixels. Combining images with contextual information about location, date, weather, habitat and user expertise has been shown to improve accuracy (de Lutio et al, 2021; Lin et al, 2021; Picek et al, 2022; Sun, Futahashi, & Yamanaka, 2021; Terry et al, 2020). Other innovations include incorporating taxonomy into the network training process (Elhamod et al, 2021; Zhang et al, 2021) and combining images with DNA sequence data (Wang, Chen, et al, 2021; Yang et al, 2022).…”

An artificial intelligence model to identify snakes from across the world: Opportunities and challenges for global health and herpetology

“…We developed part of our model collaborating with an international community of AI scientists via AICrowd and the SnakeCLEF2021 challenge [40], and building on solutions for PLOS NEGLECTED TROPICAL DISEASES classifying fungi [41]. Our model is based on Vision Transformer, the state-of-the-art deep neural network, and uses simple and replicable training procedure and unique geographic data exploitation.…”

Mapping Citizen Science through the Lens of Human-Centered AI

“…Following this global trend, the integration of ML into CitSci projects is on the rise. Specific CitSci tasks using the sub-class of ML, supervised learning (SL) for the classification of ecology images are most commonly reported on (Picek et al, 2022;Willi et al, 2018). Neural networks are also used for RNA puzzle solving (Koodli et al, 2019) as well as with respect to broader engagement and retention of CitSci volunteers (Zaken et al, 2021).…”