Automatic detection and classification of coastal Mediterranean fish from underwater images: Good practices for robust training

Abstract

Further investigation is needed to improve the identification and classification of fish in underwater images using artificial intelligence, specifically deep learning. Questions that need to be explored include the importance of using diverse backgrounds, the effect of (not) labeling small fish on precision, the number of images needed for successful classification, and whether they should be randomly selected. To address these questions, a new labeled dataset was created with over 18,400 recorded Mediterranean fish from 20 species from over 1,600 underwater images with different backgrounds. Two state-of-the-art object detectors/classifiers, YOLOv5m and Faster RCNN, were compared for the detection of the ‘fish’ category in different datasets. YOLOv5m performed better and was thus selected for classifying an increasing number of species in six combinations of labeled datasets varying in background types, balanced or unbalanced number of fishes per background, number of labeled fish, and quality of labeling. Results showed that i) it is cost-efficient to work with a reduced labeled set (a few hundred labeled objects per category) if images are carefully selected, ii) the usefulness of the trained model for classifying unseen datasets improves with the use of different backgrounds in the training dataset, and iii) avoiding training with low-quality labels (e.g., small relative size or incomplete silhouettes) yields better classification metrics. These results and dataset will help select and label images in the most effective way to improve the use of deep learning in studying underwater organisms.

Automated identification of invasive rabbitfishes in underwater images from the Mediterranean Sea

“…The recall for Siganus obtained by this model is lower than other studies on a single fish species (Ditria et al, 2020; Lopez‐Marcano et al, 2021). The overall performance on the eight species is higher than for the classification algorithm by Catalán et al (2023) for nine Mediterranean indigenous fishes (recall: 0.76, precision: 0.37) and for 47 species of tropical sharks and rays, which proposed a combination of three models (a detection model, a binary sorting model, and a classification model), achieving an overall accuracy of 70% (Jenrette et al, 2022). The two species most commonly misidentified as Siganus spp.…”

Non‐extractive fish diversity assessment in Mediterranean rhodolith beds

“…However, further development in data analysis is required. For example, the development of more automatic identification of images (e.g., Catalán et al, 2023); the use of more than one molecular marker for eDNA in order to maximize the detection success of the technique (Polanco-Fernández et al, 2021); and the development of extensive libraries of sound-producing species that are still unidentified (Di Iorio et al, 2021). This strategic focus aims to facilitate the future generation of species lists and inventories, encompassing not only fish but also invertebrates, without the necessity of destructive sampling methods.…”

Leveraging Bounding Box Annotations for Fish Segmentation in Underwater Images

“…A recent modication of YOLO, YOLOACT [31], which permits fast instance segmentation has been proposed for fish identification in [32]. Rather than focusing on the architecture, [33] analyzes the importance in the selection of an adequate dataset for robust training of the object detector. Recent reviews of the literature on underwater object detection can be found in [34], [35] and [36].…”