New Interactive Machine Learning Tool for Marine Image Analysis

Abstract

Due to advances in imaging technologies, the rate of marine video and image data collection is drastically increasing. Often these datasets are not analysed to their full potential as extracting information for multiple species, such as their presence and surface area, is incredibly time-consuming. This study demonstrates the potential of a new open-source interactive machine learning tool, RootPainter, to analyse large marine image datasets quickly and accurately. The tool was initially developed to measure plant roots, but here was tested on its ability to measure the presence and surface area of the cold-water coral reef associate sponge species,Mycale lingua, in two types of underwater image data: 18,346 time-lapse images and 1,420 remotely operated vehicle video frames. New corrective annotation metrics integrated with RootPainter, such as dice score and species area error, allow for the objective assessment of when to stop model training and reduce the need for manual model validation. Three highly accurateMycale linguamodels were created using RootPainter, as indicated by their average dice score of 0.94 ± 0.06. Model transfer and optimisation aided in the production of two of these models, increasing analysis efficiency from 6 to 16 times faster than manual annotation in Photoshop, for underwater observatory images. Sponge and surface area measurements were extracted from both datasets allowing future investigation of sponge behaviours and distributions. This study demonstrates that interactive machine learning tools and model sharing have the potential to dramatically increase image analysis speeds, collaborative research, and our collective knowledge on spatiotemporal patterns in biodiversity.

New interactive machine learning tool for marine image analysis

“…Data accessibility. The data used and models produced in this work are accessible through Pangaea [60]. Electronic supplementary material is available online [103].…”

“…RootPainter installation and model development were completed as per the GoogleColab notebook instructions [56]. A detailed manual describing model training, specifically for marine images, is also available [57].…”

“…Sponge areas were extracted from the Photoshop annotations using open-source R scripts [63] and scaled using ImageJ [62], as previously described. The precision, recall, dice score and accuracy of Model 1 were then calculated in Python [60,64], by assuming the manually annotated images were accurate. Precision is quantified as the ratio of true positives to all positive instances (the sum of true and false positives) and describes the probability that a pixel is truly foreground, given that the Root-Painter model predicts it as foreground.…”