Computational metadata generation methods for biological specimen image collections

Abstract

Metadata is a key data source for researchers seeking to apply machine learning (ML) to the vast collections of digitized biological specimens that can be found online. Unfortunately, the available metadata is often sparse and, at times, erroneous. This paper extends previous research with the Illinois Natural History Survey (INHS) collection (7,244 specimen images) using computational approaches to analyze image quality, and then automatically generate 22 metadata properties representing the image quality and morphological features of the specimens. In the research reported here, we demonstrate the extension of our initial work to University of Wisconsin Zoological Museum (UWZM) collection (4,155 specimen images). Further, we enhance our computational methods in four ways: 1) augmenting the training set, 2) applying contrast enhancement, 3) upscaling small objects, and 4) refining of our processing logic. Together these new methods improved our overall error rates from 4.6% to 1.1%. These enhancements also allowed Computational Metadata Generation Methods us to compute an additional set of 17 image-based metadata properties. The new metadata properties provide supplemental features and information that may also be used to analyze and classify the fish specimens. Examples of these new features include convex area, eccentricity, perimeter, skew, etc. The newly refined process further outperforms humans in terms of time and labor cost, as well as accuracy, providing a novel solution for leveraging digitized specimens with ML. This research demonstrates the ability of computational methods to enhance the digital library services associated with the tens of thousands of digitized specimens stored in open-access repositories world-wide by generating accurate and valuable metadata for those repositories.

From leaves to labels: Building modular machine learning networks for rapid herbarium specimen analysis with LeafMachine2

“…Few dynamic methods that rely on machine learning and computer vision techniques have been developed to automate CF determination. One study used an object detection algorithm to locate the number “2” and the number “3” on rulers to compute CFs, but was limited to only two ruler types; the authors advocated for an approach that located tick marks directly (Karnani et al, 2022 ). With LeafMachine2, we required a more generalizable procedure for obtaining image‐specific CFs.…”

A FAIR and modular image‐based workflow for knowledge discovery in the emerging field of imageomics

“…The first ML component (Figures 1 and 2, step 4a) performs object detection and metadata generation as defined by the rule 2021), and expanded upon in Karnani et al (2022). The domain scientists and software engineer worked with the ML researchers to containerize their codebase for reuse in this workflow.…”

“…MorphoSource (Boyer et al., 2016), iDigBio (http://idigbio.org) and iNaturalist (http://inaturalist.org)]. Image data are ripe for applications of ML techniques, including neural networks (NN), to extract information such as metadata (Karnani et al., 2022; Leipzig et al., 2021; Pepper et al., 2021; Rinaldo et al., 2022; Stork et al., 2019), species classification (Schuettpelz et al., 2017; Wäldchen & Mäder, 2018; Wilf et al., 2016) and presence of traits (Alfaro et al., 2019; Lürig et al., 2021; MacLeod, 2017; Weeks et al., 2016). Although ML offers powerful tools for automatic object detection and subsequent analysis of biological image data, no single ML technique provides a complete solution.…”

Challenges and opportunities in applying AI to evolutionary morphology

“…Together, species identification and taxonomic and meta-data extraction methods from images represent a powerful tool for unlocking the full potential of natural history collections. These approaches can make data more discoverable and usable for documenting biodiversity both in collections and in the field (Karnani et al, 2022;Schuettpelz et al, 2017;Wäldchen and Mäder, 2018;White et al, 2020). Information on specimens is not limited to museum catalogues but is also available in the wealth of scientific publications detailing and imaging specimens for varied purposes.…”