Head and gaze tracking of unrestrained marmosets

Abstract

New technologies for manipulating and recording the nervous system allow us to 1 perform unprecedented experiments. However, the influence of our experimental 2 manipulations on psychological processes must be inferred from their effects on 3 behavior. Today, quantifying behavior has become the bottleneck for large-scale, high 4 throughput, experiments. The method presented here addresses this issue by using deep 5 learning algorithms for video-based animal tracking. Here we describe a reliable 6 automatic method for tracking head position and orientation from simple video 7 recordings of the common marmoset (Callithrix jacchus). This method for measuring 8 marmoset behavior allows for the indirect estimation of gaze within foveal error, and 9 can easily be adapted to a wide variety of similar tasks in biomedical research. In 10 particular, the method has great potential for the simultaneous tracking of multiple 11 marmosets to quantify social behaviors. 12 Introduction 13Recent technological developments allow us to manipulate [3,35,40] and record [7,9,53] 14 the nervous system with historically unprecedented precision and scale. Yet, the 15 psychological relevance of our sophisticated manipulations and large-scale recordings 16 can only be inferred from their effect on behavior. Today, quantifying behavior has 17 become the bottleneck for high-throughput experiments [5,48]. Common practice is to 18 apply standard tests designed to measure psychological constructs such as anxiety and 19 memory, for example, by using the Elevated Plus Maze [34] or the Morris Water 20 Maze [28]. Such testing requires animals to be individually handled, making data 21 acquisition labor intensive, increasing costs and reducing experimental throughput. 22Alternatively, various simple detectors (e.g. capacitance sensors or photo-beams) can be 23 arranged to automatically acquire data at specific sites (e.g. drinking [12] and 24 feeding [10] stations). This type of automatization allows for high throughput, but fails 25 to capture complex or subtle behaviors such as social interactions and gaze behaviors. A 26 more promising approach is to record high-dimensional data from sensor arrays (e.g. 27video) and extract relevant information using computer vision algorithms. This 28 approach has the potential to provide a better characterization of behavior, capable of 29 automatically capturing complex and subtle behaviors, while simultaneously reducing 30 both cost and labor intensity [39]. Raw video frames are composed of a high number of 31 pixels whose values do not straightforwardly correlate with an animal's behavior. In

Applications for deep learning in ecology

“…Deep neural networks can automate the description of animal behaviour, thus proving valuable for ethological studies. For instance insight on the social behaviour of individuals has been gained by describing their body position and tracking their gaze (Pereira et al, ; Qiao et al, ; Turesson, Conceicao, & Ribeiro, ). Images from camera trapping have been successfully used to describe and classify wild animals’ activities such as feeding or resting (Norouzzadeh et al, ).…”

Chemogenetic activation of nigrostriatal dopamine neurons in freely moving common marmosets

“…In particular, the posture of animals -a series of movements of the major body parts -reflects both motor function and their internal state such as emotion and intention (Dael et al, 2012;Coulson, 2004;Matsumoto et al, 2013Matsumoto et al, , 2014Mimura et al, 2015;Nakamura et al, 2016). Conventional systems available for marmosets focus on simple macroscale parameters (e.g., circadian activity rhythms (Yabumoto et al, 2019)) or a few microbehavioral parameters (e.g., eye and face orientation (Brattain et al, 2016;Turesson et al, 2016)). We developed a markerless MTS that tracks and digitizes the 3D trajectory of the major body parts of marmosets, thereby reconstructing and analyzing the sequence of posture.…”

Applications for deep learning in ecology

“…Deep neural networks could prove to be valuable assets to study the behaviour of animals by providing a means to automatically describe their activities. Insight on the social behaviour of individuals could then be gained by describing their body position and tracking their gaze 33,34 . Images from camera trapping can be used to describe and classify the activities of wild animals such as feeding or resting 20 .…”

“…Diseases on crop leaves [43] Social behaviour [33] Plant phenotyping [30][31][32] Bird songs [25][26][27] Wild animals in pictures [20] Plants [22 -24]…”