A passive, camera-based head-tracking system for real-time, three-dimensional estimation of head position and orientation in rodents

Abstract

Tracking head position and orientation in small mammals is crucial for many applications in the field of behavioral neurophysiology, from the study of spatial navigation to the investigation of active sensing and perceptual representations. Many approaches to head tracking exist, but most of them only estimate the 2D coordinates of the head over the plane where the animal navigates. Full reconstruction of the pose of the head in 3D is much more more challenging and has been achieved only in handful of studies, which employed headsets made of multiple LEDs or inertial units. However, these assemblies are rather bulky and need to be powered to operate, which prevents their application in wireless experiments and in the small enclosures often used in perceptual studies. Here we propose an alternative approach, based on passively imaging a lightweight, compact, 3D structure, painted with a pattern of black dots over a white background. By applying a cascade of feature extraction algorithms that progressively refine the detection of the dots and reconstruct their geometry, we developed a tracking method that is highly precise and accurate, as assessed through a battery of validation measurements. We show that this method can be used to study how a rat samples sensory stimuli during a perceptual discrimination task and how a hippocampal place cell represents head position over extremely small spatial scales. Given its minimal encumbrance and wireless nature, our method could be ideal for high-throughput applications, where tens of animals need to be simultaneously and continuously tracked. NEW & NOTEWORTHY Head tracking is crucial in many behavioral neurophysiology studies. Yet reconstruction of the head’s pose in 3D is challenging and typically requires implanting bulky, electrically powered headsets that prevent wireless experiments and are hard to employ in operant boxes. Here we propose an alternative approach, based on passively imaging a compact, 3D dot pattern that, once implanted over the head of a rodent, allows estimating the pose of its head with high precision and accuracy.

Category learning in rodents using touchscreen‐based tasks

“…They should have equal complexity as the category stimuli; however, they should not resemble the category exemplars because this may cause interference or generalization. Controlling for changes in motor behavior can also be obtained by cameras that track head and eye movements 87,88 …”

Real-Time Closed-Loop Feedback in Behavioral Time Scales Using DeepLabCut

“…In the last 20 years, multiple research groups have reported increasingly sophisticated, easy-to-use tools for automated annotation of video data of animal behavior ( Knutsen et al, 2005 ; Voigts et al, 2008 ; Perkon et al, 2011 ; Clack et al, 2012 ; Ohayon et al, 2013 ; Giovannucci et al, 2018 ; Dominiak et al, 2019 ; Vanzella et al, 2019 ; Betting et al, 2020 ; Petersen et al, 2020 ). One natural extension of this ability has been to apply these algorithms for on-line, closed-loop paradigms, where changes in behavior of the animal are detected as rapidly as possible, and the behavior is used to modify or manipulate the brain, the virtual environment or the context of behavior.…”

“…Traditionally, video data have been analyzed manually. More recently, various algorithms have been developed for automating movement detection ( Knutsen et al, 2005 ; Voigts et al, 2008 ; Perkon et al, 2011 ; Clack et al, 2012 ; Ohayon et al, 2013 ; Giovannucci et al, 2018 ; Dominiak et al, 2019 ; Vanzella et al, 2019 ; Betting et al, 2020 ; Petersen et al, 2020 ). With the development of DeepLabCut, a marker-less pose-estimation toolkit based on deep learning ( Mathis et al, 2018 ), computer vision approaches are being used for monitoring poses of animals and for tracking the movement of virtually any part of the body.…”

Rats spontaneously perceive global motion direction of drifting plaids

“…Rats were tested using the high-throughput behavioral rig described in [15] and previously employed in several investigations of rat object recognition by our group [34,[36][37][38][39][40]. Briefly, the rig consists of six independent operant boxes, each equipped with a computer monitor for stimulus presentation and an array of three response ports for collection of behavioral responses.…”

“…The behavioral rig was the same previously used in several studies of rat visual perception carried out by our group [34,[36][37][38][39][40]. It consisted of two racks, each equipped with three operant boxes to allow training a batch of six rats simultaneously [15,35].…”