Remote sensing techniques for automated marine mammals detection: a review of methods and current challenges

Abstract

Marine mammals are under pressure from multiple threats, such as global climate change, bycatch, and vessel collisions. In this context, more frequent and spatially extensive surveys for abundance and distribution studies are necessary to inform conservation efforts. Marine mammal surveys have been performed visually from land, ships, and aircraft. These methods can be costly, logistically challenging in remote locations, dangerous to researchers, and disturbing to the animals. The growing use of imagery from satellite and unoccupied aerial systems (UAS) can help address some of these challenges, complementing crewed surveys and allowing for more frequent and evenly distributed surveys, especially for remote locations. However, manual counts in satellite and UAS imagery remain time and labor intensive, but the automation of image analyses offers promising solutions. Here, we reviewed the literature for automated methods applied to detect marine mammals in satellite and UAS imagery. The performance of studies is quantitatively compared with metrics that evaluate false positives and false negatives from automated detection against manual counts of animals, which allows for a better assessment of the impact of miscounts in conservation contexts. In general, methods that relied solely on statistical differences in the spectral responses of animals and their surroundings performed worse than studies that used convolutional neural networks (CNN). Despite mixed results, CNN showed promise, and its use and evaluation should continue. Overall, while automation can reduce time and labor, more research is needed to improve the accuracy of automated counts. With the current state of knowledge, it is best to use semi-automated approaches that involve user revision of the output. These approaches currently enable the best tradeoff between time effort and detection accuracy. Based on our analysis, we identified thermal infrared UAS imagery as a future research avenue for marine mammal detection and also recommend the further exploration of object-based image analysis (OBIA). Our analysis also showed that past studies have focused on the automated detection of baleen whales and pinnipeds and that there is a gap in studies looking at toothed whales, polar bears, sirenians, and mustelids.

Gray whale detection in satellite imagery using deep learning

“…Most ambitious would be to generate results which are informative about absolute whale density (e.g., Bamford et al., 2020), for example, to measure local abundance. In the latter cases, it is particularly important to minimize bias generated by poor precision or recall metrics (Rodofili et al., 2022). The mAP@0.5 is the metric used to judge overall model performance and is maximized by the model in this study, but consideration should be given to the importance of each metric, particularly as the confidence at which the precision and recall are calculated is different from that of the mAP.…”

“…The count deviation (3) (Rodofili et al., 2022), on the whale class, was calculated to provide a measure of the cumulative mistakes made by the model as a fraction of the total number of samples. …”

Growth and opportunities for drone surveillance in pinniped research

“…Occupied aircraft have been used to survey and estimate pinniped populations since the era of industrial sealing (Bartlett 1929), exploiting aerial perspectives to scout large regions of land or ice habitat at a time. Today, high‐resolution satellites provide even greater spatial coverage of pinniped habitats (LaRue et al 2011, Rodofili et al 2022), with advantages that include automated and relatively passive data collection, once sensors are placed in orbit, and regular coverage that depends on the satellite's orbit and revisit period, though this is reduced by coincident cloud cover. Imagery from occupied aircraft regularly achieves GSDs and quality necessary to distinguish seals in their ice or land habitats (Johnston et al 2017) and, under select circumstances, very high‐resolution satellite imagery can enable the same (LaRue et al 2017).…”

Annotating very high-resolution satellite imagery: A whale case study

“…Among the challenges to scale this technology to its full potential, is the need to analyze the imagery efficiently using automated systems, with machine learning approaches being presented as most suitable for wildlife [15 , [26] , [27] , [28] . In machine learning, models are trained to recognize and classify visual objects through an iterative process, where many examples of the target object are fed into model training [29 , 30] .…”