Automated Taxonomic Identification of Insects with Expert-Level Accuracy Using Effective Feature Transfer from Convolutional Networks

Abstract

Rapid and reliable identification of insects is important in many contexts, from the detection of disease vectors and invasive species to the sorting of material from biodiversity inventories. Because of the shortage of adequate expertise, there has long been an interest in developing automated systems for this task. Previous attempts have been based on laborious and complex handcrafted extraction of image features, but in recent years it has been shown that sophisticated convolutional neural networks (CNNs) can learn to extract relevant features automatically, without human intervention. Unfortunately, reaching expert-level accuracy in CNN identifications requires substantial computational power and huge training data sets, which are often not available for taxonomic tasks. This can be addressed using feature transfer: a CNN that has been pretrained on a generic image classification task is exposed to the taxonomic images of interest, and information about its perception of those images is used in training a simpler, dedicated identification system. Here, we develop an effective method of CNN feature transfer, which achieves expert-level accuracy in taxonomic identification of insects with training sets of 100 images or less per category, depending on the nature of data set. Specifically, we extract rich representations of intermediate to high-level image features from the CNN architecture VGG16 pretrained on the ImageNet data set. This information is submitted to a linear support vector machine classifier, which is trained on the target problem. We tested the performance of our approach on two types of challenging taxonomic tasks: 1) identifying insects to higher groups when they are likely to belong to subgroups that have not been seen previously and 2) identifying visually similar species that are difficult to separate even for experts. For the first task, our approach reached \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} }{}$CDATA[$CDATA[$>$$\end{document}92% accuracy on one data set (884 face images of 11 families of Diptera, all specimens representing unique species), and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} }{}$CDATA[$CDATA[$>$$\end{document}96% accuracy on another (2936 dorsal habitus images of 14 families of Coleoptera, over 90% of specimens belonging to unique species). For the second task, our approach outperformed a leading taxonomic expert on one data set (339 images of three species of the Coleoptera genus Oxythyrea; 97% accuracy), and both humans and traditional automated identification systems on another data set (3845 images of nine species of Plecoptera larvae; 98.6 % accuracy). Reanalyzing several biological imag...

ClassifyMe: A Field-Scouting Software for the Identification of Wildlife in Camera Trap Images

“…Animals 2020, 10, x FOR PEER REVIEW 3 of 16 more widely across agricultural and ecological monitoring [33][34][35][36][37]. In the context of camera traps it is worth noting that such algorithms have been used in prototype software for this purpose since at least 2015, in projects such as Wild Dog Alert (https://invasives.com.au/research/wild-dog-alert/) [38]-building on earlier semi-automated species recognition algorithms [39].…”

“…ClassifyMe at present is focused on species classification but future models could incorporate these additional capabilities due to the choice of YOLOv2. YOLOv2 is designed for high-throughput processing (40-90 frames per second) whilst achieving relatively high-accuracy (YOLOv2 544 × 544 mean Average Precision 78.6@49 frames per second on Pascal VOC 2007 dataset using a NVIDIA GeForce GTX Titan X GPU, [36]. A range of other competitive object detectors such as SSD [42], Faster R-CNN [43] and R-FCN [44] could also have been selected for this task.…”

“…Advances in computer vision have been pronounced in recent years, with successful demonstrations of image recognition in fields as diverse as autonomous cars, citrus tree detection from drone imagery, and the identification of skin cancer [29][30][31]. Recent work has also demonstrated the feasibility of Deep Learning approaches for species identification in camera trap images [32] and more widely across agricultural and ecological monitoring [33][34][35][36][37]. In the context of camera traps it is worth noting that such algorithms have been used in prototype software for this purpose since at least 2015, in projects such as Wild Dog Alert (https://invasives.com.au/research/wild-dog-alert/) [38]-building on earlier semi-automated species recognition algorithms [39].…”

A Convolutional Neural Network for Automatic Identification and Classification of Fall Army Worm Moth

“…(ii) Fine tuning which is trying to unfreeze a few layers from the pre-trained model and training them together with the new classification layer [16][17] [18]. Fig.…”

Mini-barcodes are equally useful for species identification and more suitable for large-scale species discovery in Metazoa than full-length barcodes

“…Traditionally, it starts with the sorting of unsorted material into major 90 taxa (e.g., order-level in insects). This task can still be accomplished by parataxonomists but 91 may in the future be taken over by machines utilizing neural networks (Valan et al, 2019). In 92 contrast, the subsequent species-level sorting is usually time-limiting because the specimens 93 for many invertebrate taxa have to be prepared and studied by highly-skilled specialists (e.g., 94dissected; slide-mounted) before the material can be sorted into putative species.…”