Collaborative Deep Learning Models to Handle Class Imbalance in FlowCam Plankton Imagery

Abstract

Using automated imaging technologies, it is now possible to generate previously unprecedented volumes of plankton image data which can be used to study the composition of plankton assemblages. However, the current need to manually classify individual images introduces a bottleneck into processing chains. Although Machine Learning techniques have been used to try and address this issue, past efforts have suffered from accuracy limitations, especially in minority classes. Here we use state-of-the-art methods in Deep Learning to investigate suitable architectures for training an automated plankton classification system which achieves high efficacy for both abundant and rare taxa. We collected live plankton from Station L4 in the Western English Channel and imaged 11,371 particles covering 104 taxonomic groups using the automated plankton imaging system FlowCam. The image set contained a severe class imbalance, with some taxa represented by > 600 images while other, rarer taxa were represented by just 14. We demonstrate that by allowing multiple Deep Learning models to collaborate in a single classification system, classification accuracy improves for minority classes when compared with the best individual model. The top collaborative model achieved a 6% improvement in F1 accuracy over the best individual model, while overall accuracy improved by 3.2%. This resulted in a 97.4% overall accuracy score and a 96.2% F1 macro score on a separate holdout test set containing 104 taxonomic groups. Based on a survey of similar studies in the literature, we believe collaborative deep learning models can significantly improve the accuracy of existing automated plankton classification systems.

Semi-automated classification of colonial Microcystis by FlowCAM imaging flow cytometry in mesocosm experiment reveals high heterogeneity during seasonal bloom

“…In the context of saving time during taxonomic analysis, imaging cytometers constitute a faster and efficient way to receive the morphological information required for taxonomic identification 5 . The imaging cytometer in our study was a FlowCAM instrument used by many research groups worldwide 5 , 8 , 12 , 19 .…”

“…For example, Lee et al 51 used the WHOI-Plankton database with 3 million plankton images, where > 90% of all images were annotated for only 5 different classes. In the recent study by Kerr and co-authors 12 , the class imbalance issue was addressed by constructing deep learning algorithms in a collaborative model to achieve the classification of under represented classes found in FlowCAM images. However, this prediction model showed poor performance in certain minority classes.…”

“…This allowed us to create class-balanced training sets by choosing time points with sufficient amounts of all five Microcystis morphospecies. It is a first attempt to apply a semi-automatic algorithm for intrageneric analysis of colonial Microcystis , the majority of previous studies being focused on the analysis of colonial phytoplankton taxa at genus level 10 , 12 , 53 or used for analysis training sets build with single-celled Microcystis laboratory cultures 9 , 11 .…”

“…Our results suggest that by combining intrageneric classification with the relatively simple set of descriptors in imaging flow cytometry, we can provide an opportunity to examine the colonial morphoforms of Microcystis at a higher resolution and temporal level during seasonal bloom. Although previous studies have developed machine learning and deep learning approach to classify plankton 1 – 12 , our study is the first to differentiate colonial morphoforms of freshwater Microcystis at the intrageneric level. The accuracy of the approach is raising to experienced human taxonomists' performance level, thereby reducing the time of analysis and subjectivity.…”

“…There is, therefore, a high demand for automating the process of classification to enable high-throughput data processing. In the last decades imaging cytometers such as FlowCytobot 6 , FlowCAM 5 , 8 , 12 , 13 , and Imagestream X Mark II 9 have been used to improve and speed up phytoplankton image acquisition. The FlowCAM instrument has become a valuable tool in marine and freshwater plankton studies because it enables researchers to classify, count, and monitor different plankton organisms 17 , 18 in the preferred detection size range of 20–300 μm 19 .…”