High-resolution animal tracking with integration of environmental information in aquatic systems

Abstract

Acquiring high resolution quantitative behavioural data underwater often involves installation of costly infrastructure, or capture and manipulation animals. Aquatic movement ecology can therefore be limited in scope of taxonomic and ecological coverage. Here we present a novel deep-learning based, multi-individual tracking approach, which incorporates Structure-from-Motion in order to determine the 3D location, body position and the visual environment of every recorded individual. The application is based on low-cost cameras and does not require the animals to be confined or handled in any way. Using this approach, single individuals, small heterospecific groups and schools of fish were tracked in freshwater and marine environments of varying complexity. Further, we established accuracy measures, resulting in positional tracking errors as low as 1.09 ± 0.47 cm (RSME) in underwater areas up to 500 m 2 . This cost-effective and open-source framework allows the analysis of animal behaviour in aquatic systems at an unprecedented resolution. Implementing this versatile approach, quantitative behavioural analysis can employed in a wide range of natural contexts, vastly expanding our potential for examining non-model systems and species.Background 1 Understanding the movement and behaviour of animals in their natural habitats is the 2 ultimate goal of behavioural and movement ecology. By situating our studies in the 3 natural world, we have the potential to uncover the processes of selection acting on the 4 behaviour in natural populations in a manner that cannot be achieved through lab 5 studies alone. The ongoing advance of animal tracking and biologging brings the 6 opportunity to revolutionize not only the scale of data collected from wild systems, but 7 also the types of questions that can subsequently be answered. Incorporating 8 geographical data has already given insights, for example, into the homing behaviour of 9 reef fish, migratory patterns of birds, or the breeding site specificity of sea 10 turtles [10,22,51]. Great advances in systems biology have further been made through 11 the study of movement ecology, understanding the decision-making processes at play 12 1/14 within primate groups manoeuvring through difficult terrain or the collective sensing of 13 birds traversing their physical environment [41,52]. Unravelling these aspects of animal 14 movement can also vastly improve management strategies [13,14], for example in the 15 creation of protected areas that incorporate bird migratory routes [48] or by reducing 16 by-catch with dynamic habitat usage models of marine turtles [37]. 17Yet the application of techniques that meet the challenges of working in naturally 18 complex environments is not straightforward, with practical, financial, and analytical 19 issues often limiting the resolution or coverage of data gathered. This is especially true 20 in aquatic ecosystems, where approaches such as Global Positioning System (GPS) tags 21 allow only sparse positioning of animals that surface more or less ...

Behavioral traits that define social dominance are the same that reduce social influence in a consensus task

“…These masks were then skeletonized into 1-pixel midlines along each mask's long axis using morphological image transformations. Subsequently, this allowed the estimation of fish spine poses (45) as seven equidistantly spaced points along these midlines. The second spine point represents an individual's head position, and the vector pointing from the second to the first spine point is its orientation.…”

“…These positional data were then used to automatically reconstruct continuous fish trajectories using a simple, distance-based identity assignment approach. Accuracy and high detection frequency were visually verified with a Python-based graphical user interface (45) developed within the laboratory that was also used to manually correct false identity assignments and losses. Mask R-CNN predictions resulted in a mean coverage of 96.3% throughout all analyzed videos and automatic trajectory assignment in an average of 14 losses per individual; 1.6% of all detections were false positives or poorly segmented, resulting in a mean coverage of 94.8% in the manually corrected trajectories.…”

SLEAP: Multi-animal pose tracking

“…DeepLabCut, LEAP, and DeepPoseKit, have all been adapted to perform multi-animal pose estimation in a top-down framework, but the process of identifying animal instances must be performed separately. More recent work using 3D data also employed top-down approaches for multi-animal pose but only in single species or specialized experimental conditions [5,15,12]. Other work has focused on bottom-up approaches using rodents, but these methods have not been shown to generalize to other types of animals [3,19].…”