From leaves to labels: Building modular machine learning networks for rapid herbarium specimen analysis with LeafMachine2

Abstract

PremiseQuantitative plant traits play a crucial role in biological research. However, traditional methods for measuring plant morphology are time consuming and have limited scalability. We present LeafMachine2, a suite of modular machine learning and computer vision tools that can automatically extract a base set of leaf traits from digital plant data sets.MethodsLeafMachine2 was trained on 494,766 manually prepared annotations from 5648 herbarium images obtained from 288 institutions and representing 2663 species; it employs a set of plant component detection and segmentation algorithms to isolate individual leaves, petioles, fruits, flowers, wood samples, buds, and roots. Our landmarking network automatically identifies and measures nine pseudo‐landmarks that occur on most broadleaf taxa. Text labels and barcodes are automatically identified by an archival component detector and are prepared for optical character recognition methods or natural language processing algorithms.ResultsLeafMachine2 can extract trait data from at least 245 angiosperm families and calculate pixel‐to‐metric conversion factors for 26 commonly used ruler types.DiscussionLeafMachine2 is a highly efficient tool for generating large quantities of plant trait data, even from occluded or overlapping leaves, field images, and non‐archival data sets. Our project, along with similar initiatives, has made significant progress in removing the bottleneck in plant trait data acquisition from herbarium specimens and shifted the focus toward the crucial task of data revision and quality control.

Model Development for Identifying Aromatic Herbs Using Object Detection Algorithm

“…Concerning the models' results for each herb and the model for the classes, there is an opportunity for improvement in the model training process for all aromatic herbs [17,18,35]. A model that does not produce false negatives, i.e., a model in which no undetected bounding boxes should be detected, has a recall of 1.0.…”

Challenges and opportunities in applying AI to evolutionary morphology

“…One area where this is proving highly effective is in automatic trait extraction from digitised herbarium specimens (Walker et al, 2022), with pilot studies have shown promising results on different types of plants. For example, in LeafMachine (Weaver et al, 2020;Weaver and Smith, 2023), a CNN algorithm was trained to measure leaf area and perimeter from lowquality images, with a success rate of 60%. In another study, a different CNN algorithm was shown to be capable of discriminating between growth shoots and vegetative structures in tropical plants from French Guiana (Goëau et al, 2022).…”

From Dormant Collections to Repositories for the Study of Habitat Changes: The Importance of Herbaria in Modern Life Sciences

“…At the same time, non-destructive specimen measurements of trace elements can be obtained using X-ray fluorescence, looking for trace metal hyperaccumulation. Interestingly, these multispectral analyses can be combined with tools based on artificial intelligence (AI) technology (Figure 2) enabling computerautomated (or semiautomated) measurement for large-scale analyses of phenological traits in thousands of specimen images [19,22,23].…”