Robust segmentation of underwater fish based on multi-level feature accumulation

Abstract

Because fish are vital to marine ecosystems, monitoring and accurate detection are crucial for assessing the potential for fisheries in these environments. Conventionally, fish-related assessment is conducted manually, which makes it labor-intensive and time-consuming. In addition, the assessments are challenging owing to underwater visibility limitations, which leads to poor detection accuracy. To overcome these problems, we propose two novel architectures for the automatic and high-performance segmentation of fish populations. In this study, the efficient fish segmentation network (EFS-Net) and multi-level feature accumulation-based segmentation network (MFAS-Net) are the base and final networks, respectively. In deep convolutional neural networks, the initial layers usually contain potential spatial information. Therefore, the EFS-Net employs a series of convolution layers in the early stage of the network for optimal feature extraction. To boost segmentation accuracy, the MFAS-Net uses an initial feature refinement and transfer block to refine potential low-level information and subsequently transfers it to the deep stages of the network. Moreover, the MFAS-Net employs multi-level feature accumulation that improves pixel-wise prediction for fish that are indistinct. The proposed networks are evaluated using two publicly available datasets, namely DeepFish and semantic segmentation of underwater imagery (SUIM), both of which contain challenging underwater fish segmentation images. The experimental results reveal that mean intersection-over-unions of 76.42% and 92.0% are attained by the proposed method for the DeepFish and SUIM datasets, respectively; these values are higher than those by the state-of-the-art methods such as A-LCFCN+PM and DPANet. In addition, high segmentation performance is achieved without compromising the computational efficiency of the networks. The MFAS-Net requires only 3.57 million trainable parameters to be fully trained. The proposed model and the complete code will be made available1.

MSGNet: multi-source guidance network for fish segmentation in underwater videos

“…Biological visual perception shows that in continuous scenes, people are easily attracted to moving objects, thus breaking the original blurred or camouflaged state. From this point, our work innovatively uses motion optical flow to help segment fish objects in underwater videos, and this is the most apparent difference between our proposed model and existing work (Haider et al, 2022;Shoffan, 2022). We first preprocess the underwater optical flow.…”

“…This approach has excellent performance for black-and-white scenes but is not practical for color images. To further improve the robustness of fish segmentation in turbid water, Haider et al (2022) presented a robust segmentation model for underwater fish based on multi-level feature accumulation, which improved the segmentation of obscure fish by using an initial feature refinement and transfer block to refine potential information. Similarly, Zhang et al (2022) employed dual pooling-aggregated attention with spatial and channel dimensions, greatly reducing the computational effort while providing better segmentation results for fuzzy fish, but the performance in complex scenarios is not known.…”

Leveraging Bounding Box Annotations for Fish Segmentation in Underwater Images

“…Thampi et al [37] apply the popular U-Net architecture [38] to the segmentation of five different fish species. In [39] two networks are used to produce the final segmentation, one of them for optimal feature extraction and the other for multi-level feature accumulation that improves the pixel-wise prediction. Recent advances in network architectures such as attention modules and Transformers have been used to capture long-range dependencies between the image pixels and improve the segmentation results [40], [41].…”

Deep Learning for Enhanced Marine Vision: Object Detection in Underwater Environments

“…Secondly, the absence of comprehensive underwater datasets hinders large-scale training and benchmarking of semantic segmentation models for general-purpose use. Existing datasets are often specific to particular applications, such as coral-reef classification [5,6] or fish detection [7,8], and lack the diversity and breadth required for broader research. Moreover, traditional class-agnostic approaches are limited to simpler tasks like foreground segmentation or obstacle detection, and they do not generalize well to multi-object semantic segmentation.…”