Deep learning-based diatom taxonomy on virtual slides

Abstract

Deep convolutional neural networks are emerging as the state of the art method for supervised classification of images also in the context of taxonomic identification. Different morphologies and imaging technologies applied across organismal groups lead to highly specific image domains, which need customization of deep learning solutions. Here we provide an example using deep convolutional neural networks (CNNs) for taxonomic identification of the morphologically diverse microalgal group of diatoms. Using a combination of high-resolution slide scanning microscopy, web-based collaborative image annotation and diatom-tailored image analysis, we assembled a diatom image database from two Southern Ocean expeditions. We use these data to investigate the effect of CNN architecture, background masking, data set size and possible concept drift upon image classification performance. Surprisingly, VGG16, a relatively old network architecture, showed the best performance and generalizing ability on our images. Different from a previous study, we found that background masking slightly improved performance. In general, training only a classifier on top of convolutional layers pre-trained on extensive, but not domain-specific image data showed surprisingly high performance (F1 scores around 97%) with already relatively few (100-300) examples per class, indicating that domain adaptation to a novel taxonomic group can be feasible with a limited investment of effort. Diatoms are microscopic algae possessing silicate shells called frustules 1. They inhabit marine and freshwater environments as well as terrestrial habitats. Taxonomic composition of their assemblages is routinely assessed using light microscopy in ecological, bioindication and paleoclimate research 2-4. Silicate frustules cleaned of organic material and embedded into high refractive index mountant on cover slips represent the most widely used type of microscopic preparations for such analyses 5,6. Attempts to computerize parts or the whole of this workflow have been made repeatedly, starting with Cairns 7 , and in the most complete manner so far by the ADIAC project 8 , motivated by the desire to speed up the taxonomic enumeration process, to reduce its dependence on highly trained taxonomic experts, and to make identification results more reproducible and transparent. Recently, we described an updated re-implementation of most parts of this workflow, covering high throughput microscopic imaging, segmentation and outline shape feature extraction of diatom specimens 9,10 which we since mainly applied in morphometric investigations 11-14. The missing component in this workflow has been automated or computer-assisted taxonomic identification. For this purpose, i.e. image classification in a taxonomic context, deep convolutional neural networks (CNNs) are currently becoming the state-of-the-art technique. Due to the broadening availability of high throughput, in part, in situ, imaging platforms 15-18 , and large publicly available image sets 19 , marine plankton has pro...

Assessing the Image Concept Drift at the OBSEA Coastal Underwater Cabled Observatory

“…Although the concept drift is a major concern for the definition of automated services, very few studies address this problem in the field of marine monitoring (Lagenkämper et al, 2020;Kloster et al, 2020). The recent literature in the computer vision and machine learning community proposes several general purpose approaches useful to mitigate the concept drift problem (Langenkämper et al, 2019).…”

“…Unfortunately, the effectiveness of such automated approaches incurs into the "concept drift" phenomenon, consisting in a progressive decrease over time of the detection and classification performance (Hashmani et al, 2019;Jameel et al, 2020;Din et al, 2021). The concept drift is largely investigated in the community of computer vision and artificial intelligence, but very few contributions are available in the marine science context (Langenkämper et al, 2020;Kloster et al, 2020).…”

Novel Approach to Freshwater Diatom Profiling and Identification Using Raman Spectroscopy and Chemometric Analysis

“…Libreros and colleagues employed 16,000 segments of 365 images, combined with ANN, to identify diatom genera, achieving a classification accuracy of 74% [57]. A more recent study also using ANN and based on virtual slides obtained through high resolution focus-enhanced light microscopic slide scanning and a series of imaging processing steps, achieved a 95% identification accuracy of four diatom species (Fragilariopsis kerguelensis, Fragilariopsis rhombica, Thalassiosira gracilis, Thalassiosira lentiginosa) and five diatom genera (Asteromphalus, Chaetoceros, Nitzschia, Pseudonitzschia, Rhizosolenia) using a total of 2977 specimens [17]. According to these authors, around 100 specimens per taxon are required for this excellent identification.…”

“…For this reason, a great effort has recently been directed to developing faster and less cumbersome identification methods and metrics. These are mainly based either on DNA metabarcoding or a combination of diatom imaging acquisition and deep learning methods [13][14][15][16][17][18][19][20][21][22][23][24][25][26][27]. While these alternatives show promising results and take advantage of state-of-the-art sequencing and imaging methods, they are still laborious and quite expensive, which limits their application to routine monitoring of water quality.…”

Current Trends and Future Directions of Large Scale Image and Video Annotation: Observations From Four Years of BIIGLE 2.0

“…This became apparent when BIIGLE users requested the support for virtual microscopy slides-large images in the gigapixel range-in the image annotation tool. Support for gigapixel image annotation was implemented in BIIGLE in late 2017 and already supported several studies in the marine biology community (Kloster et al, 2020;Burfeid-Castellanos et al, 2021). Gigapixel images differ from regular, smaller images since the original image files cannot be displayed unmodified in a web browser.…”