Benchmarking and Automating the Image Recognition Capability of an In Situ Plankton Imaging System

Abstract

To understand ocean health, it is crucial to monitor photosynthetic marine plankton – the microorganisms that form the base of the marine food web and are responsible for the uptake of atmospheric carbon. With the recent development of in situ microscopes that can acquire vast numbers of images of these organisms, the use of deep learning methods to taxonomically identify them has come to the forefront. Given this, two questions arise: 1) How well do deep learning methods such as Convolutional Neural Networks (CNNs) identify these marine organisms using data from in situ microscopes? 2) How well do CNN-derived estimates of abundance agree with established net and bottle-based sampling? Here, using images collected by the in situ Scripps Plankton Camera (SPC) system, we trained a CNN to recognize 9 species of phytoplankton, some of which are associated with Harmful Algal Blooms (HABs). The CNNs evaluated on 26 independent natural samples collected at Scripps Pier achieved an averaged accuracy of 92%, with 7 of 10 target categories above 85%. To compare abundance estimates, we fit a linear model between the number of organisms of each species counted in a known volume in the lab, with the number of organisms collected by the in situ microscope sampling at the same time. The linear fit between lab and in situ counts of several of the most abundant key HAB species suggests that, in the case of dinoflagellates, there is good correspondence between the two methods. As one advantage of our method, given the excellent correlation between lab counts and in situ microscope counts for key species, the methodology proposed here provides a way to estimate an equivalent volume in which the employed microscope can identify in-focus organisms and obtain statistically robust estimates of abundance.

Environmental and ecological drivers of harmful algal blooms revealed by automated underwater microscopy

“…We refer to this measurement as abundance (cells d −1 ), though we note that, because the sample volume is not well constrained, our measure of abundance is not a traditional volumetric measure of cells per unit volume. This measurement unit used here linearly corresponds to manual plankton counts taken at Scripps Pier (Le et al 2022).…”

Deep focus-extended darkfield imaging for in situ observation of marine plankton

“…Based on our experience in IPP nearshore deployment, this will result in a total water volume enlargement from 388.8L to 2592L per day at an imaging frame rate of 3FPS. This is expected to further fill the gap in seawater sampling throughput and plankton quantification between the traditional methods and the in situ imagers (Barth and Stone, 2022;Le et al, 2022).…”

“…Rotermund and Samson, 2015;Gallager, 2019;Orenstein et al, 2020;Li et al, 2022), through which digital images of plankters are captured in natural seawaters. By further analysis of the obtained images using digital processing and machine learning algorithms, people can achieve automatic observation of plankton taxonomy and various functional traits (Orenstein et al, 2022). Compared with traditional methods, in situ imaging has the advantages of longer observational time and continuity, and higher spatio-temporal resolution.…”

Robust detection of marine life with label-free image feature learning and probability calibration

“…(Lumini and Nanni 2019) trained convolutional networks on many labeled datasets, then tuned the networks further on the task of interest. Other studies instead initialized via supervised training on ImageNet data (Kyathanahally et al 2021, Le et al 2022. Here we will take a different approach using semi-supervised learning.…”