Computer-vision object tracking for monitoring bottlenose dolphin habitat use and kinematics

Abstract

This research presents a framework to enable computer-automated observation and monitoring of bottlenose dolphins (Tursiops truncatus) in a zoo environment. The resulting approach enables detailed persistent monitoring of the animals that is not possible using manual annotation methods. Fixed overhead cameras were used to opportunistically collect ∼100 hours of observations, recorded over multiple days, including time both during and outside of formal training sessions, to demonstrate the viability of the framework. Animal locations were estimated using convolutional neural network (CNN) object detectors and Kalman filter post-processing. The resulting animal tracks were used to quantify habitat use and animal kinematics. Additionally, Kolmogorov-Smirnov analyses of the swimming kinematics were used in high-level behavioral mode classification. The object detectors achieved a minimum Average Precision of 0.76, and the post-processed results yielded 1.24 × 107 estimated dolphin locations. Animal kinematic diversity was found to be lowest in the morning and peaked immediately before noon. Regions of the zoo habitat displaying the highest activity levels correlated to locations associated with animal care specialists, conspecifics, or enrichment. The work presented here demonstrates that CNN object detection is viable for large-scale marine mammal tracking, and results from the proposed framework will enable future research that will offer new insights into dolphin behavior, biomechanics, and how environmental context affects movement and activity.

An Integrated Framework with ADD-LSTM and DeepLabCut for Dolphin Behavior Classification

“…Gabaldon et al [24] proposed a framework based on deep learning to monitor and analyze dolphin behavior in artificially controlled environments. They used convolutional neural networks (CNNs) and the Faster R-CNN algorithm to detect dolphins in video footage.…”

“…These techniques use RPCA for background subtraction and compressed tracking algorithms for automated dolphin tracking, which deliver a precision rate of 78.8% in dolphin identification [23]. Another approach is to use neural networks like Faster R-CNN and Kalman filters for dolphin recognition and trajectory generation, which achieve an accuracy rate of 81% in dolphin identification [24].…”

Relation between beluga whale aggregations and sea temperature on climate change forecasts

“…In remote sensing methods, free Google Earth images have been used to train Deep Convolutional Neural Networks (CNN) for conservation goals (Guirado et al, 2017), which automatically learn the distinctive features of each object class from a large set of annotated images (LeCun et al, 2015). Previous studies in marine mammal detection and counting have resulted in a recall rate of more than 80% (Bogucki et al, 2019;Guirado et al, 2019;Gabaldon et al, 2022;Khan et al, 2022). This can have a big impact in reducing the effort required for manual verification, increasing the advantage of employing an automatic detector in long-term monitoring (Li et al, 2022;Marquez et al, 2022), estimating population abundances and projecting dynamics and fluctuations under future climate change scenarios.…”

Case study of detection and monitoring of wildlife by UAVs equipped with RGB camera and TIR camera

“…It is notable that observing the natural world allows a range of relevant information to be extracted from the field of biomechanics, increasing the body of knowledge about the movement of individual animal species and insects. Gabaldon et al ’s (2022) study presents an analysis of the application of a convolutional neural network in the observation of the movement of bottlenose dolphins ( Tursiops truncatus ) in a zoo environment. Basu et al ’s (2019) paper describes how the speed of galloping giraffes was measured using unmanned aerial vehicles (UAVs), when measured with sensors mounted on a drone.…”