Machine learning techniques to characterize functional traits of plankton from image data

Abstract

Plankton imaging systems supported by automated classification and analysis have improved ecologists' ability to observe aquatic ecosystems. Today, we are on the cusp of reliably tracking plankton populations with a suite of lab-based and in situ tools, collecting imaging data at unprecedentedly fine spatial and temporal scales. But these data have potential well beyond examining the abundances of different taxa; the individual images themselves contain a wealth of information on functional traits. Here, we outline traits that could be measured from image data, suggest machine learning and computer vision approaches to extract functional trait information from the images, and discuss promising avenues for novel studies. The approaches we discuss are data agnostic and are broadly applicable to imagery of other aquatic or terrestrial organisms.1 Note that in this paper "morphology" should be understood in its biological sense, that is, the visually identifiable properties of an object, rather than in its computer vision sense, that is, the numerical characteristics derived from the binary mask of the object (its "imprint" in the image).

Content-Aware Segmentation of Objects Spanning a Large Size Range: Application to Plankton Images

“…Although CNN-based object detection may seem overwhelming at first, both in terms of set up and processing time, it actually is fast enough and within the reach of marine ecologists, particularly now that artificial intelligence frameworks and GPU computing are being made more accessible. This work constitutes a step towards the "intelligent" segmentation of ecological images, even at low resolution, which could find even wider applications such as the automated separation of objects overlapping onto each other on an image for more accurate species counts, the detection and classification in a single step for more automated surveys, or the extraction of individual-level traits to track e.g., reproductive organs development, for a richer exploitation of ecological images (Orenstein et al, 2021). Such tasks are in no way limited to plankton images and are common in data collected by trawl cameras, benthic observations or surveying cameras, vessel monitoring cameras, etc.…”

Demystifying image-based machine learning: A practical guide to automated analysis of field imagery using modern machine learning tools

“…Many of the traditional problems marine scientists currently use imagery to address fall into one of three categories: image classification, object detection, or semantic segmentation. More complex tasks such as tracking (Irisson et al 2022, Katija et al 2021, functional trait analysis (Orenstein et al 2022), pose estimation , and automated measurements (Fernandes et al 2020) often rely on these more basic tasks as building blocks. In what follows, we will focus primarily on these three core tasks, and reference other applications of image-based ML where appropriate.…”

“…2022, Katija et al . 2021), functional trait analysis (Orenstein et al . 2022), pose estimation (Mathis et al .…”

Comparison of an In Situ Imaging Device and Net-Based Method to Study Mesozooplankton Communities in an Oligotrophic System

“…In the past several years, plankton ecologists are increasingly utilizing a trait-based approach to characterize zooplankton communities (Litchman et al, 2013;Kiørboe et al, 2018). Recently, there has been advances in combining in situ imaging and trait-based methodology to study zooplankton (Ohman, 2019;Vilgrain et al, 2021;Orenstein et al, 2021). Studying plankton in situ is particularly important for the study of fragile and gelatinous organisms.…”