Improving plankton image classification using context metadata

Abstract

Advances in both hardware and software are enabling rapid proliferation of in situ plankton imaging methods, requiring more effective machine learning approaches to image classification. Deep Learning methods, such as convolutional neural networks (CNNs), show marked improvement over traditional feature-based supervised machine learning algorithms, but require careful optimization of hyperparameters and adequate training sets. Here, we document some best practices in applying CNNs to zooplankton and marine snow images and note where our results differ from contemporary Deep Learning findings in other domains. We boost the performance of CNN classifiers by incorporating metadata of different types and illustrate how to assimilate metadata beyond simple concatenation. We utilize both geotemporal (e.g., sample depth, location, time of day) and hydrographic (e.g., temperature, salinity, chlorophyll a) metadata and show that either type by itself, or both combined, can substantially reduce error rates. Incorporation of context metadata also boosts performance of the feature-based classifiers we evaluated: Random Forest, Extremely Randomized Trees, Gradient Boosted Classifier, Support Vector Machines, and Multilayer Perceptron. For our assessments, we use an original data set of 350,000 in situ images (roughly 50% marine snow and 50% nonsnow sorted into 26 categories) from a novel in situ Zooglider. We document asymptotically increasing performance with more computationally intensive techniques, such as substantially deeper networks and artificially augmented data sets. Our best model achieves 92.3% accuracy with our 27-class data set. We provide guidance for further refinements that are likely to provide additional gains in classifier accuracy.

Seasonal Depletion of the Dissolved Iron Reservoirs in the Sub‐Antarctic Zone of the Southern Atlantic Ocean

“…The isopycnal depth of deep winter mixing was identified at a density threshold of 26.70 kg/m 3 and extended throughout summer to represent remnant winter waters. Depth-integrated PP rates (PP wc , mol C·m −2 ·day −1 ) were calculated from quenching corrected glider-derived chlorophyll (Thomalla, Moutier, et al, 2017) and PAR according to Platt et al (1980), Platt and Sathyendranath (1993), and Thomalla et al (2015). PP parameters were determined from a linear relationship with chlorophyll using experimental values from both cruises (supporting information Figure S2; Ryan-Ryan-Keogh, .…”

Thinking like a naturalist: Enhancing computer vision of citizen science images by harnessing contextual data

“…This approach can greatly reduce the risk of non‐sensical identifications that otherwise lead to considerable scepticism over the use of automated methods (Gaston & O'Neill, ). Nevertheless, this ‘filtering’ approach does not make full use the available data, and it has been recently shown that improvements in the identification of plankton from images can be improved by incorporating sample metadata directly into a neural network (Ellen, Graff, & Ohman, ). Many species vary in appearance seasonally or across their range.…”

Bayesian Models for Deriving Biogeochemical Information from Satellite Ocean Color

“…In a similar vein, a number of in-water studies have investigated similar approaches using NNs to estimate Chl, IOPs, apparent optical properties (AOPs) and water constituent concentrations for optically complex (i.e. case 2) waters (24)(25)(26)(27)(28) (29) or plankton taxonomic studies (30), where the abundance of data enables the use of state-of-the art approaches such as deep learning. Ocean color remote sensing, on the other hand, su ers from a severe lack of labelled data (3), i.e.…”