Murine Motion Behavior Recognition Based on DeepLabCut and Convolutional Long Short-Term Memory Network

Abstract

Murine behavior recognition is widely used in biology, neuroscience, pharmacology, and other aspects of research, and provides a basis for judging the psychological and physiological state of mice. To solve the problem whereby traditional behavior recognition methods only model behavioral changes in mice over time or space, we propose a symmetrical algorithm that can capture spatiotemporal information based on behavioral changes. The algorithm first uses the improved DeepLabCut keypoint detection algorithm to locate the nose, left ear, right ear, and tail root of the mouse, and then uses the ConvLSTM network to extract spatiotemporal information from the keypoint feature map sequence to classify five behaviors of mice: walking straight, resting, grooming, standing upright, and turning. We developed a murine keypoint detection and behavior recognition dataset, and experiments showed that the method achieved a percentage of correct keypoints (PCK) of 87±1% at three scales and against four backgrounds, while the classification accuracy for the five kinds of behaviors reached 93±1%. The proposed method is thus accurate for keypoint detection and behavior recognition, and is a useful tool for murine motion behavior recognition.

An Integrated Framework with ADD-LSTM and DeepLabCut for Dolphin Behavior Classification

“…Liu, Ruiqing, Juncai Zhu, and Xiaoping Rao [17] conducted a study on identifying mouse behavior using an improved DeepLabCut network for keypoint detection and behavior recognition through convolutional long short-term memory (ConvLSTM) networks. The authors utilized an enhanced DeepLabCut keypoint detection algorithm to identify specific points on the mouse's body, such as the nose, ears, and tail base.…”

“…This module initially uses a pose estimation model to perform a preliminary analysis of the processed dolphin key points, categorizing dolphin behaviors into six types based on set thresholds, namely vomit fish, side swimming, swimming together, playing with the toys, resting by the shore, and hook the swimming ring. Finally, a double-layer bidirectional LSTM recurrent neural network [16,17] is used to further classify dolphin behaviors, achieving a behavior recognition accuracy of 94.3%.…”

Image recognition method for dangerous behavior of non-stop construction personnel in large airports

“…Reference [8] puts forward a method of cow image behavior recognition based on deep learning, which divides the data into different categories related to the most important activities in the image, and uses convolutional neural network classifier to identify cow behavior. Reference [9] proposes a symmetric algorithm to capture spatio-temporal information based on behavior changes, uses the improved DeepLabCut key point detection algorithm to locate mouse limbs, and uses ConvLSTM network to extract spatio-temporal information from key point feature map sequence to classify mouse behaviors, thus completing the recognition of mouse movement behaviors. However, the accuracy of the above methods in feature extraction is not high, which leads to the unsatisfactory accuracy of identifying dangerous behavior images.…”

AI‐Enabled Micro Motion Sensors for Revealing the Random Daily Activities of Caged Mice

“…One commonly used approach is to apply computer vision techniques to analyze the recorded videos and classify mouse behaviors. [12,13] Another approach, which is discussed in this paper, is to capture mouse motion (e.g., x-, y-, z-accelerations and angular velocities) directly by micro-inertial measurement unit (μIMU) sensors. Due to the advancements in microelectromechanical systems (MEMS) technology in the past two decades, the small-size and considerably low-cost μIMUs have led to the popularity of applying these sensing devices in sectors such as robotics, sports, and navigation.…”