DeepWild: Application of the pose estimation tool DeepLabCut for behaviour tracking in wild chimpanzees and bonobos

Abstract

Studying animal behaviour allows us to understand how different species and individuals navigate their physical and social worlds. Video coding of behaviour is considered a gold standard: allowing researchers to extract rich nuanced behavioural datasets, validate their reliability, and for research to be replicated. However, in practice, videos are only useful if data can be efficiently extracted. Manually locating relevant footage in 10,000 s of hours is extremely time‐consuming, as is the manual coding of animal behaviour, which requires extensive training to achieve reliability. Machine learning approaches are used to automate the recognition of patterns within data, considerably reducing the time taken to extract data and improving reliability. However, tracking visual information to recognise nuanced behaviour is a challenging problem and, to date, the tracking and pose‐estimation tools used to detect behaviour are typically applied where the visual environment is highly controlled. Animal behaviour researchers are interested in applying these tools to the study of wild animals, but it is not clear to what extent doing so is currently possible, or which tools are most suited to particular problems. To address this gap in knowledge, we describe the new tools available in this rapidly evolving landscape, suggest guidance for tool selection, provide a worked demonstration of the use of machine learning to track movement in video data of wild apes, and make our base models available for use. We use a pose‐estimation tool, DeepLabCut, to demonstrate successful training of two pilot models of an extremely challenging pose estimate and tracking problem: multi‐animal wild forest‐living chimpanzees and bonobos across behavioural contexts from hand‐held video footage. With DeepWild we show that, without requiring specific expertise in machine learning, pose estimation and movement tracking of free‐living wild primates in visually complex environments is an attainable goal for behavioural researchers.

Application of a novel deep learning–based 3D videography workflow to bat flight

“…Bat flight represents a particularly difficult challenge for deep learning networks compared to most animal movements for which the method has been used 12–18 . During flight, bats can be filmed from all directions in both the horizontal and vertical planes.…”

“…Artificial intelligence (AI)–based methods for measuring animal behavior, including locomotor movement, have recently made great strides. For example, the widely used Python package DeepLabCut (DLC) 11 has been successfully applied to humans, 12 rats, 13 cheetahs, 14 macaques, 15 chimpanzees, 16 spiders, 17 and more 18 . In contrast, DLC has not been widely applied to animals in flight 3 .…”

“Technical Note: DeepLabCut-Display: open-source desktop application for visualizing and analyzing two-dimensional locomotor data in livestock”

“…Wiltshire et al used DLC to make pose estimates of chimpanzees and bonobos. They found that the machine learning models produced pose estimations with a consistency similar or better than many different human labeling (Wiltshire et al, 2023), and in a fraction of the time required for human labeling. The model was also able to work in a variety of environments, from open clearings to dense forest.…”

A toolkit for the dynamic study of air sacs in siamang and other elastic circular structures

“…Animal skeletal pose tracking from video data has undergone nothing short of a revolution through developments in computer vision and deep learning [1][2][3][4][5][6]. There are now many pretrained pose detection models for humans, and there are increasingly more pre-trained models for non-human animals (e.g., rhesus macaques, Macaca mulatta, [7], chimpanzees, Pan troglodytes [8,9]). These developments are key for animal research as they allow non-invasive monitoring based on video alone, which can further be used for automatic classification of behavioral patterns [10][11][12].…”