On the impact of Citizen Science-derived data quality on deep learning based classification in marine images

Abstract

The evaluation of large amounts of digital image data is of growing importance for biology, including for the exploration and monitoring of marine habitats. However, only a tiny percentage of the image data collected is evaluated by marine biologists who manually interpret and annotate the image contents, which can be slow and laborious. In order to overcome the bottleneck in image annotation, two strategies are increasingly proposed: “citizen science” and “machine learning”. In this study, we investigated how the combination of citizen science, to detect objects, and machine learning, to classify megafauna, could be used to automate annotation of underwater images. For this purpose, multiple large data sets of citizen science annotations with different degrees of common errors and inaccuracies observed in citizen science data were simulated by modifying “gold standard” annotations done by an experienced marine biologist. The parameters of the simulation were determined on the basis of two citizen science experiments. It allowed us to analyze the relationship between the outcome of a citizen science study and the quality of the classifications of a deep learning megafauna classifier. The results show great potential for combining citizen science with machine learning, provided that the participants are informed precisely about the annotation protocol. Inaccuracies in the position of the annotation had the most substantial influence on the classification accuracy, whereas the size of the marking and false positive detections had a smaller influence.

Gear-Induced Concept Drift in Marine Images and Its Effect on Deep Learning Classification

“…Furthermore, changes in these communities across the spatial and/or temporal domains can be recorded. First attempts to link these two developments have been successful and showed the potential of deep learning in e.g., morphotype detection (Zurowietz et al, 2018), morphotype classification (Smith and Dunbabin, 2007;Gobi, 2010;Beijbom et al, 2012;Bewley et al, 2012;Kavasidis and Palazzo, 2012;Schoening et al, 2012;Langenkämper et al, 2018Langenkämper et al, , 2019Mahmood et al, 2019;Piechaud et al, 2019) or polyp behavior monitoring (Osterloff et al, 2019). However, all these studies have reported results obtained for data sets collected with the same gear, i.e., with one distinct camera system and the platform for the full analyzed data set.…”

ALMI—A Generic Active Learning System for Computational Object Classification in Marine Observation Images

“…In almost all works published, these images are in fact image patches, marked by domain experts in large images showing an underwater scenery containing multiple objects. The detection and extraction of these patches showing single objects can be done by experts, sometimes supported by computational methods that often employ unsupervised learning [ 14 , 15 , 16 ] or even citizen scientists [ 17 ]. However, one task that cannot be supported straightforwardly with computational methods or non-experts is the final classification of objects to taxonomic categories or morphotypes, and this task is addressed in this work.…”

Using Patent Technology Networks to Observe Neurocomputing Technology Hotspots and Development Trends

“…Neurocomputing originally referred to hardware that mimics neuroscience structures to create models of the nervous system [1]. This concept is further extended to computing systems that operate using bioinspired computing models, including neural networks [2] and deep-learning networks [3]. In recent years, widespread research on neurocomputing technology has been driven by the rapid development of cognitive learning applications and the limited computing power of the Von Neumann architecture.…”