Marker-less tracking system for multiple mice using Mask R-CNN

Abstract

Although the appropriate evaluation of mouse behavior is crucial in pharmacological research, most current methods focus on single mouse behavior under light conditions, owing to the limitations of human observation and experimental tools. In this study, we aimed to develop a novel marker-less tracking method for multiple mice with top-view videos using deep-learning-based techniques. The following stepwise method was introduced: (i) detection of mouse contours, (ii) assignment of identifiers (IDs) to each mouse, and (iii) correction of mis-predictions. The behavior of C57BL/6 mice was recorded in an open-field arena, and the mouse contours were manually annotated for hundreds of frame images. Then, we trained the mask regional convolutional neural network (Mask R-CNN) with all annotated images. The mouse contours predicted by the trained model in each frame were assigned to IDs by calculating the similarities of every mouse pair between frames. After assigning IDs, correction steps were applied to remove the predictive errors semi-automatically. The established method could accurately predict two to four mice for first-look videos recorded under light conditions. The method could also be applied to videos recorded under dark conditions, extending our ability to accurately observe and analyze the sociality of nocturnal mice. This technology would enable a new approach to understand mouse sociality and advance the pharmacological research.

Open Pose Mask R-CNN Network for Individual Cattle Recognition

“…Other studies have proposed improved Mask R-CNN detection and segmentation algorithms, such as Dai et al [57], who increased the third channel of the FPN feature extraction path to obtain more comprehensive feature information and improve the accuracy of the segmentation mask. Sakamoto et al [58] developed a new label free tracking method to train Mask R-CNN networks with all annotated images. Devi et al [59] combined Mask R-CNN and YOLOv3 for mixed livestock classification and early detection of unintended activities.…”

IntelliCage: the development and perspectives of a mouse- and user-friendly automated behavioral test system

“…Nonetheless, once achieved an effective motion tracking system, there will be newly developed software packages dealing with the immense data sets recorded, chiefly based on artificial intelligence (AI) including machine learning, such as DeepLabCut which enables body point estimation and tracking of individual animals housed singly or in a group ( Mathis et al, 2018 ; Nath et al, 2019 ). There is an increasing number of AI-based technologies available to track and further analyse fine body movements according to trained classifiers for mouse behavior which are objective and independent of human definitions ( Nilsson et al, 2020 ; Dunn et al, 2021 ; Fong et al, 2023 ; Sakamoto et al, 2023 ). Thus, one may expect further support by video techniques for the fine-grained analysis of mouse behavior in an IntelliCage, but the amount of data analysis associated with it (the so-called data footprint) will require careful consideration of specific experimental questions to be asked.…”