Automated Semantic Annotation of Species Names in Handwritten Texts

Abstract

In this paper, scientific species names from images of handwritten species observations are automatically recognised and annotated with semantic concepts, so that they can be used for document retrieval and faceted search. Until now, automated semantic annotation of such named entities was only applied to printed or digital text. We employ a two-step approach. First, word images are classified, identifying elements of scientific species names; Genus, species, author, using (i) visual structural features, (ii) position, and (iii) context. Second, the identified species names are semantically annotated according to the NHC-Ontology, an ontology that describes species observations. Internationalised Resource Identifiers (IRIs) are assigned to the elements so that they can be linked and disambiguated at a later stage by individual researchers. For the identification of scientific species names, we achieve an average F1 score of 0.86. Moreover, we discuss how our method will function in a semi-automated annotation process, with a fruitful dialogue between system and user as the main objective.

An End-to-End, Interactive Deep Learning Based Annotation System for Cursive and Print English Handwritten Text

“…Similarly, with the evident need for annotated data for offline handwritten text, especially with the shift towards Deep Learning based approaches for offline handwriting text recognition, it is not surprising that there has been a lot of research in this area too. There have been quite a few works that have proposed a systematic arrangement of stages to create a complete annotation engine for handwritten text, comprising of varying levels of automation [8][9]. However, a complete end-to-end pipeline to annotate handwritten text with very minimal human interaction is still considered to be a very challenging task as discussed in a part of a study by Ung et al [10].…”

A FAIR and modular image‐based workflow for knowledge discovery in the emerging field of imageomics

“…MorphoSource (Boyer et al., 2016), iDigBio (http://idigbio.org) and iNaturalist (http://inaturalist.org)]. Image data are ripe for applications of ML techniques, including neural networks (NN), to extract information such as metadata (Karnani et al., 2022; Leipzig et al., 2021; Pepper et al., 2021; Rinaldo et al., 2022; Stork et al., 2019), species classification (Schuettpelz et al., 2017; Wäldchen & Mäder, 2018; Wilf et al., 2016) and presence of traits (Alfaro et al., 2019; Lürig et al., 2021; MacLeod, 2017; Weeks et al., 2016). Although ML offers powerful tools for automatic object detection and subsequent analysis of biological image data, no single ML technique provides a complete solution.…”