SealNet 2.0: Human-Level Fully-Automated Pack-Ice Seal Detection in Very-High-Resolution Satellite Imagery with CNN Model Ensembles

Abstract

Pack-ice seals are key indicator species in the Southern Ocean. Their large size (2–4 m) and continent-wide distribution make them ideal candidates for monitoring programs via very-high-resolution satellite imagery. The sheer volume of imagery required, however, hampers our ability to rely on manual annotation alone. Here, we present SealNet 2.0, a fully automated approach to seal detection that couples a sea ice segmentation model to find potential seal habitats with an ensemble of semantic segmentation convolutional neural network models for seal detection. Our best ensemble attains 0.806 precision and 0.640 recall on an out-of-sample test dataset, surpassing two trained human observers. Built upon the original SealNet, it outperforms its predecessor by using annotation datasets focused on sea ice only, a comprehensive hyperparameter study leveraging substantial high-performance computing resources, and post-processing through regression head outputs and segmentation head logits at predicted seal locations. Even with a simplified version of our ensemble model, using AI predictions as a guide dramatically boosted the precision and recall of two human experts, showing potential as a training device for novice seal annotators. Like human observers, the performance of our automated approach deteriorates with terrain ruggedness, highlighting the need for statistical treatment to draw global population estimates from AI output.

Species classification of Antarctic pack‐ice seals using very high‐resolution imagery

“…Given the low frequency of these misclassifications, it is challenging to ascertain whether they stem from limitations in the model or annotation errors by the analysts (i.e., incorrectly labeling a seal's species). While misclassifications are commonplace in machine learning models, this finding emphasizes the importance of two elements: (1) augmenting the size and range of the training and validation data sets to evaluate the model's performance more comprehensively (Gonçalves et al 2022); and (2) the complexity of creating and validating such models when direct, ground‐truthing is either impossible or impractical.…”

“…Antarctic pack-ice seals haul out on a variety of sea ice environments. To characterize and extract pack-ice (free floating sea ice) and fast-ice (sea ice "fastened" to the continent) environmental features, we segmented individual image scenes using the CNN pipeline developed in Gonçalves et al (2022). Features identified by the CNN segmentation as sea ice were automatically extracted as geospatial polygon features and exported as an GeoJSON layer, with geographic coordinate information appended to each ice floe (Figure 3).…”

“…While these studies provided valuable insight into seal abundance and spatial distribution, manually annotating pack-ice seals in VHR imagery is extremely laborious and challenged by variations in lighting, substrate conditions, and postprocessing artifacts (Gonçalves et al, 2020). More recently, the use of deep learning and computer vision methods has enabled researchers to identify seals across whole imagery catalogs in an automated manner (Gonçalves et al, 2020(Gonçalves et al, , 2022. The development of methodology to automatically detect seals from VHR imagery have allowed researchers to quickly process the large amount of imagery gathered over the Southern Ocean, providing a platform for comprehensive continental-scale survey campaigns.…”

Through the looking-glass : Marine mammal monitoring in a changing world

“…As the number of training images was relatively low, it is expected that the performance could increase by training the network with more images and seals. Later versions of their approach (Gonçalves et al 2022) include a sea ice segmentation model to first find potential suitable seal habitats. However, when using VHR satellite imagery, it becomes extremely challenging to detect pups, which are smaller and generally better camouflaged (e.g., Fudala and Bialik, 2022) and to differentiate between sympatric pinniped species (e.g., Gonçalves et al, 2020).…”

“…This can dramatically reduce the time and cost necessary to process the images, but the use of non-expert observers caused a high rate of false positives (LaRue et al, 2020). Alternatively, Gonçalves et al (2020) applied an automated approach to detect pinnipeds from satellite imagery, which was later improved by including a sea ice segmentation model (Gonçalves et al, 2022). Both studies were unable to differentiate between four sympatric species, but still serve as first examples of the rapid recent developments in automated wildlife detection, thereby lowering the costs and time to process the images dramatically.…”