Hierarchical Classification of Very Small Objects: Application to the Detection of Arthropod Species

Tresson, Paul; Carval, Dominique; Tixier, Philippe; Puech, William

doi:10.1109/access.2021.3075293

Cited by 11 publications

(6 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, those relationships are hierarchical in nature, and hierarchical classification has been researched in different application domains (Silla and Freitas, 2011;Salakhutdinov et al, 2013;Park and Kim, 2020) such as diatom images (Dimitrovski et al, 2012), disease detection (An et al, 2021) and protein families (Sandaruwan and Wannige, 2021). For animals such as arthropods (Tresson et al, 2021) and fish species (Gupta et al, 2022) hierarchical classification has been investigated with the object detector YOLOv3 (Redmon and Farhadi, 2018) designed to detect and classify species using a "flat" multi-class structure. Tresson et al (2021) used a two-step approach with a YOLOv3 object detector to detect and classify arthropods in five super-classes, using a separate model to classify cropped images at the species level.…”

Section: Background and Related Workmentioning

confidence: 99%

Hierarchical Classification of Insects with Multitask Learning and Anomaly Detection

Bjerge¹,

Geissmann

Alison

et al. 2023

Preprint

View full text Add to dashboard Cite

Cameras and computer vision are revolutionising the study of insects, creating new research opportunities within agriculture, epidemiology, evolution, ecology and monitoring of biodiversity. However, a major challenge is the diversity of insects and close resemblances of many species combined with computer vision are often not sufficient to classify large numbers of insect species, which sometimes cannot be identified at the species level. Here, we present an algorithm to hierarchically classify insects from images, leveraging a simple taxonomy to (1) classify specimens across multiple taxonomic ranks simultaneously, and (2) highlight the lowest rank at which a reliable classification can be reached. Specifically, we propose multitask learning, a loss function incorporating class dependency at each taxonomic rank, and anomaly detection based on outlier analysis for quantification of uncertainty. First, we compile a dataset of 41,731 images of insects, combining images from time-lapse monitoring of floral scenes with images from the Global Biodiversity Information Facility (GBIF). Second, we adapt state-of-the-art convolutional neural networks, ResNet and EfficientNet, for the hierarchical classification of insects belonging to three orders, five families and nine species. Third, we assess model generalization for 11 species unseen by the trained models. Here, anomaly detection is used to predict the higher rank of the species not present in the training set. We found that incorporating a simple taxonomy into our model increased accuracy at higher taxonomic ranks. As expected, our algorithm correctly classified new insect species at higher taxonomic ranks, while classification was uncertain at lower taxonomic ranks. Anomaly detection can effectively flag novel taxa that are visually distinct from species in the training data. However, five novel taxa were consistently mistaken for visually similar species in the training data. Above all, we have demonstrated a practical approach to hierarchical classification based on species taxonomy and uncertainty during automated in situ monitoring of live insects. Our method is simple and versatile and could be implemented to classify a wide range of insects as well as other organisms.

show abstract

Section: Background and Related Workmentioning

confidence: 99%

Hierarchical Classification of Insects with Multitask Learning and Anomaly Detection

Bjerge¹,

Geissmann

Alison

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Drastic changes in arthropod population abundance and diversity have negative cascading effects on ecological stability and ecosystem resiliency (Borer et al, 2012; Kremen et al, 1993; Tscharntke et al, 2012). To expedite and improve the analysis of these trends, the ecological field is currently developing deep learning methods to better understand this potential threat of food web collapse (Arje, Melvad, et al, 2020; Helton et al, 2022; Schneider et al, 2022; Tresson et al, 2021; Wani & Maul, 2021).…”

Section: Introductionmentioning

confidence: 99%

“…Deep learning in the field of entomology continually makes strides to accomplish tasks that previously required human experts (Hansen et al, 2020;Xin et al, 2020). This is particularly true for classification and detection (Ramcharan et al, 2017;Tresson et al, 2021) where deep learning models have, in recent years, standardized around specific vision architectures (ResNet, DenseNet, Vision Transformer, etc.) (Dosovitskiy et al, 2020;Gao et al, 2018;Szegedy et al, 2017).…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Getting the bugs out of AI: Advancing ecological research on arthropods through computer vision

et al. 2023

View full text Add to dashboard Cite

Deep learning for computer vision has shown promising results in the field of entomology, however, there still remains untapped potential. Deep learning performance is enabled primarily by large quantities of annotated data which, outside of rare circumstances, are limited in ecological studies. Currently, to utilize deep learning systems, ecologists undergo extensive data collection efforts, or limit their problem to niche tasks. These solutions do not scale to region agnostic models. However, there are solutions that employ data augmentation, simulators, generative models, and self-supervised learning that can supplement limited labelled data. Here, we highlight the success of deep learning for computer vision within entomology, discuss data collection efforts, provide methodologies for optimizing learning from limited annotations, and conclude with practical guidelines for how to achieve a foundation model for entomology capable of accessible automated ecological monitoring on a global scale.

show abstract

“…Drastic changes in arthropod population abundance and diversity have negative cascading effects on ecological stability and ecosystem resiliency [8,9,10]. To expedite and improve the analysis of these trends, the ecological field is currently developing deep learning methods to better understand this potential threat of food web collapse [11,12,13,14,15].…”

Section: Introductionmentioning

confidence: 99%

Getting the Bugs Out: Entomology Using Computer Vision

Schneider

Taylor

Kremer

et al. 2022

Preprint

View full text Add to dashboard Cite

Deep learning for computer vision has shown promising results in the field of entomology. Deep learning performance is maximized primarily by bulk labeled data which, outside of rare circumstances, are limited in ecological studies. Currently, to utilize deep learning systems, ecologists undergo extensive data collection efforts, or limit their problem to niche tasks. These solutions do not scale to region agnostic models. There are solutions using data augmentation, simulators, generative models, and self-supervised learning that supplement limited data labels. Here, we highlight the success of deep learning for computer vision within entomology, discuss data collection efforts, provide methodologies for annotation efficient learning, and conclude with practical guidelines for how ecologists can empower accessible automated ecological monitoring on a global scale.

show abstract

Hierarchical Classification of Very Small Objects: Application to the Detection of Arthropod Species

Abstract: We would like to thank the following organisations for their support, ANR-16-CONV-0004 #DigitAg as well as CIRAD DPP COSAQ agronomical research programme (activities 2015-2021) funded by a grant from the European Community (ERDF) and the Conseil Régional de La Réunion.

Cited by 11 publications

References 19 publications

Hierarchical Classification of Insects with Multitask Learning and Anomaly Detection

Hierarchical Classification of Insects with Multitask Learning and Anomaly Detection

Getting the bugs out of AI: Advancing ecological research on arthropods through computer vision

Getting the Bugs Out: Entomology Using Computer Vision

Contact Info

Product

Resources

About