Biodiversity Image Quality Metadata Augments Convolutional Neural Network Classification of Fish Species

Abstract

Biodiversity image repositories are crucial sources for training machine learning approaches to support biological research. Metadata about object (e.g. image) quality is a putatively important prerequisite to selecting samples for these experiments. This paper reports on a study demonstrating the importance of image quality metadata for a species classification experiment involving a corpus of 1935 fish specimen images which were annotated with 22 metadata quality properties. A small subset of high quality images produced an F1 accuracy of 0.41 compared to 0.35 for a taxonomically matched subset low quality images when used by a convolutional neural network approach to species identification. Using the full corpus of images revealed that image quality differed between correctly classified and misclassified images. We found anatomical feature visibility was the most important quality feature for classification accuracy. We suggest biodiversity image repositories consider adopting a minimal set of image quality metadata to support machine learning.

Automatic Metadata Generation for Fish Specimen Image Collections

“…Technicians employed by Tulane University have manually generated the 22 metadata properties deemed crucial to the overall BGNN project [9] for a large number of INHS images. 20, 699 total entries were created by 13 technicians that spanned 8, 398 unique images, of which 7, 244 were both not part of the training set and met our current admissibility criteria for and pixel processing.…”

“…Using ML and image informatics algorithms, it is able to locate, mask and analyze specimens (currently limited to fish) in collection images with a high degree of accuracy. It produces 6 of the 22 core BGNN metadata properties [9], as well as image contrast, bounding boxes, scale and length information. Testing this approach on 7, 244 images from the INHS dataset [4], we see that the vast majority of the resulting metadata is correct within a tolerance of a few percentage points, and in some cases contains fewer mistakes than the manually generated validation data.…”

“…Even so, metadata describing image quality is severely limited and generally missing. This point is addressed in detail by Leipzig et al [9] and serves as the rationale for Tulane University’s effort to manually capture content for 22 metadata properties that characterize digitized specimen image quality. Their work is being conducted in connection with the larger BGNN initiative, and the difficulties encountered during the process underscore the need to explore automatic metadata generation methods.…”

Untitled

“…Species identifications and relationships among species have traditionally been based on color patterns, although some scutellation characters have been used (Bailey, 1970). Actually, physical vouchers remain the main standard in biological research in terms of reproducibility and permanence (Ceríaco et al 2016;Buckner et al 2021), However, there are cases where there is no physical voucher due to circumstances such as is a protected species or too rare to obtain permits, does not exist, is difficult to collect, among other cases; photographs and molecular data can be useful (Buckner et al 2021;Leipzig et al 2021). Especially if there is more data associated, such as coordinates and date and time information (Leipzig et al 2021).…”