Classification of Underwater Fish Images and Videos via Very Small Convolutional Neural Networks

Abstract

The automatic classification of fish species appearing in images and videos from underwater cameras is a challenging task, albeit one with a large potential impact in environment conservation, marine fauna health assessment, and fishing policy. Deep neural network models, such as convolutional neural networks, are a popular solution to image recognition problems. However, such models typically require very large datasets to train millions of model parameters. Because underwater fish image and video datasets are scarce, non-uniform, and often extremely unbalanced, deep neural networks may be inadequately trained, and undergo a much larger risk of overfitting. In this paper, we propose small convolutional neural networks as a practical engineering solution that helps tackle fish image classification. The concept of “small” refers to the number of parameters of the resulting models: smaller models are lighter to run on low-power devices, and drain fewer resources per execution. This is especially relevant for fish recognition systems that run unattended on offshore platforms, often on embedded hardware. Here, established deep neural network models would require too many computational resources. We show that even networks with little more than 12,000 parameters provide an acceptable working degree of accuracy in the classification task (almost 42% for six fish species), even when trained on small and unbalanced datasets. If the fish images come from videos, we augment the data via a low-complexity object tracking algorithm, increasing the accuracy to almost 49% for six fish species. We tested the networks with images obtained from the deployments of an experimental system in the Mediterranean sea, showing a good level of accuracy given the low quality of the dataset.

Automatic detection and classification of coastal Mediterranean fish from underwater images: Good practices for robust training

“…To increase the use of DL in this field, we concur with other authors that not only should common databases and reproducible methods be made available (e.g., Francescangeli et al, 2023), but also that more integrated engineers-ecologists interactions are institutionally needed (Logares et al, 2021). Additionally, statistical corrections to DL estimates must be developed (Connolly et al, 2021) and the use of lighter networks (e.g., Paraschiv et al, 2022) should become more common, as computer power may be a significant limitation for unplugged underwater devices (e.g., Lisani et al, 2012). In summary, our research has discovered or reinforced several key findings that have important implications for fish ecology.…”

“…Studies have demonstrated the capabilities of these techniques, particularly deep convolutional networks (CNN; LeCun et al, 1998;Lecun et al, 2015) in detecting and classifying fish in underwater images or video streams (Salman et al, 2016;Villon et al, 2018, see reviews in Goodwin et al, 2022;Li and Du, 2022;Mittal et al, 2022;Saleh, Sheaves and Rahimi Azghadi, 2022). These studies have utilized different types of image databases and have faced similar unresolved questions, such as the number of fish needed for training ( Marrable et al, 2022), the need for color image pre-processing (e.g., Lisani et al, 2022), the need for transfer learning from large databases (e.g., Imagenet or coco), improving results when working with small image areas or limited computing power (Paraschiv et al, 2022), whether to use segmentation of bounding boxes and how well a trained set will perform for different habitats (backgrounds). In particular, the detection and classification of multiple species using different combinations of backgrounds (the "domain-shift" phenomenon: Kalogeiton et al, 2016;Ditria et al, 2020), number of species, and labeling quality, is an area that requires further investigation.…”

Classification Of Some Edible Submarine Organisms in The Mediterranean Region by Species Using Deep Learning Methods

“…In general, there aren't many photographs that are related to the classes of interest, especially when you consider that they have to be taken underwater. As a result, supplements derived from various datasets have been created [17]. Table 1 presents the datasets, algorithms, performance metrics and years of study used by similar studies in the literature.…”

DAMNet: Dual Attention Mechanism Deep Neural Network for Underwater Biological Image Classification

“…Yang et al [28] proposed an initial attention network that outperformed other networks in distinguishing underwater images from images in non-underwater environments with 99.3% classification accuracy. Paraschiv et al [29] used a lightweight CNN model to classify underwater fish, which improved the accuracy by 7% compared with the network model with many parameters. Mathur et al [30] proposed a migration learning based method for underwater image classification, which improved the classification accuracy by training only the last few layers of the network through migration learning and achieved 98.44% and 84.92% accuracy on large and small datasets, respectively.…”