A Layered, Hybrid Machine Learning Analytic Workflow for Mouse Risk Assessment Behavior

Abstract

Accurate and efficient quantification of animal behavior facilitates the understanding of the brain. An emerging approach within machine learning (ML) field is to combine multiple ML-based algorithms to quantify animal behavior. These so-called hybrid models have emerged because of limitations associated with supervised (e.g., random forest, RF) and unsupervised (e.g., hidden Markov model, HMM) ML models. For example, RF models lack temporal information across video frames, and HMM latent states are often difficult to interpret. We sought to develop a hybrid model, and did so in the context of a study of mouse risk assessment behavior. We utilized DeepLabCut to estimate the positions of mouse body parts. Positional features were calculated using DeepLabCut outputs and were used to train RF and HMM models with equal number of states, separately. The per-frame predictions from RF and HMM models were then passed to a second HMM model layer ("reHMM"). The outputs of the reHMM layer showed improved interpretability over the initial HMM output. Finally, we combined predictions from RF and HMM models with selected positional features to train a third HMM model ("reHMM+"). This reHMM+ layered hybrid model unveiled distinctive temporal and human-interpretable behavioral patterns. We applied this workflow to investigate risk assessment to trimethylthiazoline and snake feces odor, finding unique behavioral patterns to each that were separable from attractive and neutral stimuli. We conclude that this layered, hybrid ML workflow represents a balanced approach for improving the depth and reliability of ML classifiers in chemosensory and other behavioral contexts.Significance StatementIn this study, we integrate two widely-adopted machine learning (ML) models, random forest and hidden Markov model, to develop a layered, hybrid ML-based workflow. Our workflow not only overcomes the intrinsic limitations of each model alone, but also improves the depth and reliability of ML models. Implementing this analytic workflow unveils distinctive and dynamic mouse behavioral patterns to chemosensory cues in the context of mouse risk assessment behavioral experiments. This study provides an efficient and interpretable analytic strategy for the quantification of animal behavior in diverse experimental settings.

Can MiceGPT be True? A Survey on AI-empowered Mice Behavior Analysis Applications and Solutions

“…Martins et al [31] collected videos of the tail suspension test (TST) in a controlled environment. Wang et al [32] collect mice behavior with an overhead camera during video recording.…”

“…Wang et al [32] seek to develop a hybrid machine learning workflow to understand the brain more by accurate and effective quantification of animal behavior. They use DeepLabCut to trace the mice body key points and detect the mice behavior during a video period by random forest and hidden Markov model models.…”

“…Neurobehavioral Assessment [54] Semantic Segmentation, Key Point Detection SV-T [52] Image Classification SV-T [55] Image Classification SV-T [10] Semantic Segmentation, Action Recognition SV-T [56] Semantic Segmentation, Image Classification SV-F [53] Action Prediction SV-F [57] Semantic Segmentation SV-F Social Behavior Analysis [39] Object Detection, Action Recognition SV-S [47] Pose Estimation, Action Recognition MV-TFS [46] Pose Estimation, Action Recognition Object Tracing SV-T [40] Action Recognition, Key Point Detection MV-TS [48] Action Recognition MV-TS [38] Object Detection, Pose Estimation, Action recognition MV-TF External Stimuli Effective Assessment [25] Key Point Detection, Action Recognition MV-B [26] Pose Estimation SV-B [27] Object Detection, Semantic Segmentation, Image Classification SV-F [28] Action Recognition SV-T [29] Instance Segmentation, Key Point Detection, Object Tracing, Action Recognition SV-T [30] Action Recognition SV-T [31] Object Detection, Action Recognition SV-F [32] Object Tracing, Action Recognition SV-T Disease Detection [18] Semantic Segmentation, Action Recognition SV-B [16] Semantic Segmentation, Action Recognition SV-T [19] Key Point Detection, Pose Estimation MV-BS [20] Key Point Detection, Pose Estimation SV-S…”

“…Wang et al [32] and Winters et al [46] both use DeepLabCut to track mice behaviors. DeepLabCut is an open-source software package for markerless pose estimation of animals and humans in video data using deep learning, which can be used for tracking the pose of mice.…”

“…The DeepLabCut architecture is a deep neural network based on ResNet, referred to as a "multi-residual network". Wang et al [32] use DeepLabCut to estimate the positions of mouse body parts. Positional features are calculated using DeepLabCut outputs and are used to train random forest and hidden markov models with equal number of states, separately.…”