Geometric deep learning enables 3D kinematic profiling across species and environments

Dunn, Timothy W; Marshall, Jesse D.; Severson, Kyle S.; Aldarondo, Diego E.; Hildebrand, David G. C.; Chettih, Selmaan N; Wang, William L.; Gellis, Amanda J; Carlson, David; Aronov, Dmitriy; Freiwald, Winrich A.; Wang, Fan; Ölveczky, Bence P.

doi:10.1038/s41592-021-01106-6

Cited by 150 publications

(161 citation statements)

References 60 publications

Supporting

Mentioning

157

Contrasting

Order By: Relevance

“…This suggests that behavioral readouts may indicate biological interventions at a level more sensitive than detectable even by the human eye (Wiltschko et al, 2020). Recent advances in deep learning for image processing have boosted the field to new heights, especially with the publication of DeepLabCut (Mathis et al, 2018), SLEAP (Pereira et al, 2019(Pereira et al, , 2020 and DANNCE (Dunn et al, 2021) for the estimation of body points. Follow-up modules B-Soid (Hsu and Yttri, 2019) and SimBA (Nilsson et al, 2020) are now available to train behavior classifiers on body-part pose data.…”

Section: Today and The Futurementioning

confidence: 99%

Measuring Behavior in the Home Cage: Study Design, Applications, Challenges, and Perspectives

Grieco

Bernstein

Biemans

et al. 2021

Front. Behav. Neurosci.

View full text Add to dashboard Cite

The reproducibility crisis (or replication crisis) in biomedical research is a particularly existential and under-addressed issue in the field of behavioral neuroscience, where, in spite of efforts to standardize testing and assay protocols, several known and unknown sources of confounding environmental factors add to variance. Human interference is a major contributor to variability both within and across laboratories, as well as novelty-induced anxiety. Attempts to reduce human interference and to measure more "natural" behaviors in subjects has led to the development of automated home-cage monitoring systems. These systems enable prolonged and longitudinal recordings, and provide large continuous measures of spontaneous behavior that can be analyzed across multiple time scales. In this review, a diverse team of neuroscientists and product developers share their experiences using such an automated monitoring system that combines Noldus PhenoTyper® home-cages and the video-based tracking software, EthoVision® XT, to extract digital biomarkers of motor, emotional, social and cognitive behavior. After presenting our working definition of a “home-cage”, we compare home-cage testing with more conventional out-of-cage tests (e.g., the open field) and outline the various advantages of the former, including opportunities for within-subject analyses and assessments of circadian and ultradian activity. Next, we address technical issues pertaining to the acquisition of behavioral data, such as the fine-tuning of the tracking software and the potential for integration with biotelemetry and optogenetics. Finally, we provide guidance on which behavioral measures to emphasize, how to filter, segment, and analyze behavior, and how to use analysis scripts. We summarize how the PhenoTyper has applications to study neuropharmacology as well as animal models of neurodegenerative and neuropsychiatric illness. Looking forward, we examine current challenges and the impact of new developments. Examples include the automated recognition of specific behaviors, unambiguous tracking of individuals in a social context, the development of more animal-centered measures of behavior and ways of dealing with large datasets. Together, we advocate that by embracing standardized home-cage monitoring platforms like the PhenoTyper, we are poised to directly assess issues pertaining to reproducibility, and more importantly, measure features of rodent behavior under more ethologically relevant scenarios.

show abstract

Section: Today and The Futurementioning

confidence: 99%

Measuring Behavior in the Home Cage: Study Design, Applications, Challenges, and Perspectives

Grieco

Bernstein

Biemans

et al. 2021

Front. Behav. Neurosci.

View full text Add to dashboard Cite

show abstract

“…On one hand, the study of behavioural correlates linked to neural activity is giving previously unimaginable insights. The possibility to look with sub-second resolution at communities and transitions between different behavioral states allows to correlate single cell firing and oscillations to the animals' action with an unprecedented time resolution (Hong et al, 2015;Wei and Kording, 2018;Luxem et al, 2020;Dunn et al, 2021;Hausmann et al, 2021). Moreover, unsupervised approaches based on machine learning algorithms to score behaviour are replacing manual scoring, which is intrinsically prone to subjective biases and therefore produces results that are hard to compare between studies.…”

Section: Future Directions and Open Questionsmentioning

confidence: 99%

The Role of the Medial Septum—Associated Networks in Controlling Locomotion and Motivation to Move

Mocellin

Mikulovic

2021

Front. Neural Circuits

View full text Add to dashboard Cite

The Medial Septum and diagonal Band of Broca (MSDB) was initially studied for its role in locomotion. However, the last several decades were focussed on its intriguing function in theta rhythm generation. Early studies relied on electrical stimulation, lesions and pharmacological manipulation, and reported an inconclusive picture regarding the role of the MSDB circuits. Recent studies using more specific methodologies have started to elucidate the differential role of the MSDB’s specific cell populations in controlling both theta rhythm and behaviour. In particular, a novel theory is emerging showing that different MSDB’s cell populations project to different brain regions and control distinct aspects of behaviour. While the majority of these behaviours involve movement, increasing evidence suggests that MSDB-related networks govern the motivational aspect of actions, rather than locomotion per se. Here, we review the literature that links MSDB, theta activity, and locomotion and propose open questions, future directions, and methods that could be employed to elucidate the diverse roles of the MSDB-associated networks.

show abstract

“…Some machine learning algorithms have shown high precision in object detection, such as deformable parts models (Felzenszwalb et al, 2010;Unger et al, 2017), R-CNN (Girshick et al, 2014), and deep neural networks (Geuther et al, 2019;Yoon et al, 2019). Using those algorithms, several toolboxes were developed for precisely calculating the postures of laboratory animals during movements, such as DeepLabCut (Mathis et al, 2018;Nath et al, 2019), LEAP (Wang et al, 2017;Pereira et al, 2019), DeepPoseKit (Graving et al, 2019), TRex (Walter and Couzin, 2021), and DANNCE (Dunn et al, 2021;Karashchuk et al, 2021), which greatly simplify and speed up the analysis of multiple behaviors. Although these algorithms may be used to track the gross movement of an animal, they are timeconsuming and insufficiently accurate for our purposes because the exact centers cannot be obtained in the process of creating a training dataset.…”

Section: Introductionmentioning

confidence: 99%

DeepBhvTracking: A Novel Behavior Tracking Method for Laboratory Animals Based on Deep Learning

Sun

Lyu

Cai

et al. 2021

Front. Behav. Neurosci.

View full text Add to dashboard Cite

Behavioral measurement and evaluation are broadly used to understand brain functions in neuroscience, especially for investigations of movement disorders, social deficits, and mental diseases. Numerous commercial software and open-source programs have been developed for tracking the movement of laboratory animals, allowing animal behavior to be analyzed digitally. In vivo optical imaging and electrophysiological recording in freely behaving animals are now widely used to understand neural functions in circuits. However, it is always a challenge to accurately track the movement of an animal under certain complex conditions due to uneven environment illumination, variations in animal models, and interference from recording devices and experimenters. To overcome these challenges, we have developed a strategy to track the movement of an animal by combining a deep learning technique, the You Only Look Once (YOLO) algorithm, with a background subtraction algorithm, a method we label DeepBhvTracking. In our method, we first train the detector using manually labeled images and a pretrained deep-learning neural network combined with YOLO, then generate bounding boxes of the targets using the trained detector, and finally track the center of the targets by calculating their centroid in the bounding box using background subtraction. Using DeepBhvTracking, the movement of animals can be tracked accurately in complex environments and can be used in different behavior paradigms and for different animal models. Therefore, DeepBhvTracking can be broadly used in studies of neuroscience, medicine, and machine learning algorithms.

show abstract

Geometric deep learning enables 3D kinematic profiling across species and environments

Cited by 150 publications

References 60 publications

Measuring Behavior in the Home Cage: Study Design, Applications, Challenges, and Perspectives

Measuring Behavior in the Home Cage: Study Design, Applications, Challenges, and Perspectives

The Role of the Medial Septum—Associated Networks in Controlling Locomotion and Motivation to Move

DeepBhvTracking: A Novel Behavior Tracking Method for Laboratory Animals Based on Deep Learning

Contact Info

Product

Resources

About