An Open Source Unsupervised Algorithm for Identification and Fast Prediction of Behaviors

Abstract

The motivation, control, and selection of actions comprising naturalistic behaviors remains a tantalizing but difficult field of study. Detailed and unbiased quantification is critical. Interpreting the positions of animals and their limbs can be useful in studying behavior, and significant recent advances have made this step straightforward (1, 2). However, body position alone does not provide a grasp of the dynamic range of naturalistic behaviors. Behavioral Segmentation of Open-field In DeepLabCut, or B-SOiD ("B-side"), is an unsupervised learning algorithm that serves to discover and classify behaviors that are not pre-defined by users. Our algorithm segregates statistically different, sub-second rodent behaviors with a single bottom-up perspective video camera. Upon DeepLabCut estimating the positions of 6 body parts (snout, the 4 paws, and the base of the tail), our software performs novel expectation maximization fitting of Gaussian mixture models on t-Distributed Stochastic Neighbor Embedding (t-SNE). The original features taken from dimensionally-reduced classes are then used build a multi-class support vector machine classifier that can decode millions of actions within seconds. We demonstrate that the highly reproducible, independently-classified behaviors can be used to extract kinematic parameters of individual actions as well as broader action sequences. This open-source platform enables the efficient study of the neural mechanisms of spontaneous behavior as well as the performance of disease-related behaviors that have been difficult to quantify, such as grooming and stride-length in Obsessive-Compulsive Disorder (OCD) and stroke research. Behavioral Analysis | Open-field | DeepLabCut | Unsupervised LearningCorrespondence: eyttri@andrew.cmu.edu

A Hierarchical 3D-motion Learning Framework for Animal Spontaneous Behavior Mapping

“…Our framework has two main advantages. First, our approach of tracking multiple body parts and acquiring 3D reconstruction data achieves better performance than similar recently reported rodent behavioral recognition frameworks [11,57]. The high signal-to-noise ratio of the data yielded by the present method avoids animal body occlusion and view-angle bias in single-camera top-view monitoring.…”

“…Meanwhile, automated and high-throughput quantification and description of animal behavior are becoming increasingly popular [6,7]. The recent emergence of automated animal pose estimation toolboxes has dramatically facilitated body parts tracking [8][9][10], specific well-defined behaviors (e.g., grooming, locomotion) can thus be detected based on body features with supervised approaches [11]. However, most naturalistic rodent behaviors are highly complex and variable; The labor-intensive, repetitive and biased manual labeling is insufficient to produce high-quality training sets [12,13].…”

DeepEthogram: a machine learning pipeline for supervised behavior classification from raw pixels

“…In addition, it is possible that specialized pipelines designed specifically for one task, such as mouse social behaviors 7 , could perform better at that specific task. An alternate approach to ours is to use innovative methods for estimating pose, including DeepLabCut 22,23 , LEAP 27 , and others 25 , followed by frame-by-frame classification of behaviors based on pose in a supervised 7,19,21 or unsupervised 17 way. Using pose for classification could make behavior classifiers faster to train and less susceptible to overfitting.…”

“…This task is distinct from other emerging behavioral analysis methods based on unsupervised learning, in which machine learning models discover behavioral modules from the data, irrespective of researcher labels. Although unsupervised methods, such as Motion Sequencing 13,14 , MotionMapper 15 , BehaveNet 16 , B-SOiD 17 , and others 8 , can discover behavioral modules not obvious to the researcher, their outputs can be challenging to match up to behaviors of interest in cases in which researchers have strong prior knowledge about the specific behaviors relevant to their experiments.…”

Deep-learning based identification, tracking, pose estimation, and behavior classification of interacting primates and mice in complex environments

“…Although this has been an impressive feat for the field, a key element, the direct recognition of behavior itself has been rarely addressed. Unsupervised analysis of behavior 9,10 can be a powerful tool to capture the diversity of the underlying behavioral patterns, but the results of these methods do not align with human annotations and therefore require subsequent inspection. Here we demonstrate a complementary approach for researchers who seek to automatically identify particular behaviors of interest.…”