Rat Behavior Observation System Based on Transfer Learning

Jin, Tianlei; Duan, Feng

doi:10.1109/access.2019.2916339

Cited by 7 publications

(4 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The accurate quantification and manipulation of behavioral dynamics of animals is important for understanding the neural basis of motor function (Jin and Duan, 2019;Moreira et al, 2019). Real-time movement tracking is a challenging computer vision problem that is crucial for constructing precise movement-triggered feedback systems and brain-machine interfaces needed for mechanistic studies of animal behavior.…”

Section: Introductionmentioning

confidence: 99%

Real-Time Selective Markerless Tracking of Forepaws of Head Fixed Mice Using Deep Neural Networks

Forys

Xiao

Gupta

et al. 2020

eNeuro

View full text Add to dashboard Cite

Compare paw positions in real timeIf threshold met: Trigger feedback (latency of 30-45 ms) y position at t 1 t = 1 t = 0 y position at t 0 Threshold: |y position at t 1 -y position at t 0 | >= 5 pixels Left pawHere, we describe a system capable of tracking specific mouse paw movements at high frame rates (70.17 Hz) with a high level of accuracy (mean = 0.95, SD , 0.01). Short-latency markerless tracking of specific body parts opens up the possibility of manipulating motor feedback. We present a software and hardware scheme built on DeepLabCut-a robust movement-tracking deep neural network framework-which enables real-time estimation of paw and digit movements of mice. Using this approach, we demonstrate movement-generated feedback by triggering a USB-GPIO (general-purpose input/output)-controlled LED when the movement of one paw, but not the other, selectively exceeds a preset threshold. The mean time delay between paw movement initiation and LED flash was 44.41 ms (SD = 36.39 ms), a latency sufficient for applying behaviorally triggered feedback. We adapt DeepLabCut for real-time tracking as an open-source package we term DeepCut2RealTime. The ability of the package to rapidly assess animal behavior was demonstrated by reinforcing specific movements within water-restricted, head-fixed mice. This system could inform future work on Significance StatementWe present a software and hardware scheme modified from DeepLabCut-a robust movement-tracking deep neural network framework-which enables real-time estimation of paw and digit movements of mice. Coupled to the body part tracking is the ability to rapidly trigger external events such as rewards on the detection of specific behaviors. This system lays the groundwork for a behaviorally triggered "closed loop" brain-machine interface that could reinforce behaviors and deliver feedback to brain regions based on prespecified body movements.

show abstract

Section: Introductionmentioning

confidence: 99%

Real-Time Selective Markerless Tracking of Forepaws of Head Fixed Mice Using Deep Neural Networks

Forys

Xiao

Gupta

et al. 2020

eNeuro

View full text Add to dashboard Cite

show abstract

“…Recently, there is a great interest in using machine learning and neural networks to understand social behavior [9][10][11]. Lorbach in [12] presents a rat social interaction dataset with results of using it as a training set for a method of recognizing interactions with rats.…”

Section: Introductionmentioning

confidence: 99%

Detection and Model of Thermal Traces Left after Aggressive Behavior of Laboratory Rodents

et al. 2021

View full text Add to dashboard Cite

Automation of complex social behavior analysis of experimental animals would allow for faster, more accurate and reliable research results in many biological, pharmacological, and medical fields. However, there are behaviors that are not only difficult to detect for the computer, but also for the human observer. Here, we present an analysis of the method for identifying aggressive behavior in thermal images by detecting traces of saliva left on the animals’ fur after a bite, nape attack, or grooming. We have checked the detection capabilities using simulations of social test conditions inspired by real observations and measurements. Detection of simulated traces different in size and temperature on single original frame revealed the dependence of the parameters of commonly used corner detectors (R score, ranking) on the parameters of the traces. We have also simulated temperature of saliva changes in time and proved that the detection time does not affect the correctness of the approximation of the observed process. Furthermore, tracking the dynamics of temperature changes of these traces allows to conclude about the exact moment of the aggressive action. In conclusion, the proposed algorithm together with thermal imaging provides additional data necessary to automate the analysis of social behavior in rodents.

show abstract

“…These methods, however, necessitate complex surgery or special markers to achieve the desired results (Burgos-Artizzu et al, 2012;Ohayon et al, 2013;Eftaxiopoulou et al, 2014;Maghsoudi et al, 2017). In the past 2 years, the observation of rat behavior based on deep neural networks has greatly improved the robustness of the observation results without the need for invasive sensors or markers (Mathis et al, 2018;Jin and Duan, 2019). Although these rat behavior observation methods are all macroscopic, which solves the problem of the rat's location and the rat's behavior at a specific time point, they do not reveal how the rat is moving.…”

Section: Introductionmentioning

confidence: 99%

“…Specifically, we designed a special running machine for the rat and used a frame rate camera to capture its motion. For the estimation process, we used our previous work on rat observation to detect the rat's position (Jin and Duan, 2019). Moreover, in order to discern the landmark points including the eyes and joints, we designed two different cascade neural networks with three different coordinate calculation methods.…”

Section: Introductionmentioning

confidence: 99%

Markerless Rat Behavior Quantification With Cascade Neural Network

et al. 2020

Self Cite

View full text Add to dashboard Cite

Quantifying rat behavior through video surveillance is crucial for medicine, neuroscience, and other fields. In this paper, we focus on the challenging problem of estimating landmark points, such as the rat's eyes and joints, only with image processing and quantify the motion behavior of the rat. Firstly, we placed the rat on a special running machine and used a high frame rate camera to capture its motion. Secondly, we designed the cascade convolution network (CCN) and cascade hourglass network (CHN), which are two structures to extract features of the images. Three coordinate calculation methods-fully connected regression (FCR), heatmap maximum position (HMP), and heatmap integral regression (HIR)-were used to locate the coordinates of the landmark points. Thirdly, through a strict normalized evaluation criterion, we analyzed the accuracy of the different structures and coordinate calculation methods for rat landmark point estimation in various feature map sizes. The results demonstrated that the CCN structure with the HIR method achieved the highest estimation accuracy of 75%, which is sufficient to accurately track and quantify rat joint motion.

show abstract

Rat Behavior Observation System Based on Transfer Learning

Cited by 7 publications

References 39 publications

Real-Time Selective Markerless Tracking of Forepaws of Head Fixed Mice Using Deep Neural Networks

Real-Time Selective Markerless Tracking of Forepaws of Head Fixed Mice Using Deep Neural Networks

Detection and Model of Thermal Traces Left after Aggressive Behavior of Laboratory Rodents

Markerless Rat Behavior Quantification With Cascade Neural Network

Contact Info

Product

Resources

About