MacaquePose: A novel ‘in the wild’ macaque monkey pose dataset for markerless motion capture

Abstract

Video-based markerless motion capture permits quantification of an animal’s pose and motion, with a high spatiotemporal resolution in a naturalistic context, and is a powerful tool for analyzing the relationship between the animal’s behaviors and its brain functions. Macaque monkeys are excellent non-human primate models, especially for studying neuroscience. Due to the lack of a dataset allowing training of a deep neural network for the macaque’s markerless motion capture in the naturalistic context, it has been challenging to apply this technology for macaques-based studies. In this study, we created MacaquePose, a novel open dataset with manually labeled body part positions for macaques in naturalistic scenes, consisting of >13,000 images, refined by researchers. We show that the pose estimation performance of an artificial neural network trained with the dataset is close to that of a human-level. The MacaquePose will provide a platform for innovative behavior analysis for non-human primate.

Deep learning based behavioral profiling of rodent stroke recovery

“…One of the latest contributions to this toolbox is the opensource software DeepLabCut (DLC) 16 , which uses convolutional neural networks to automatically capture movements and postures directly from images and without requiring active or passive markers. DLC is a modified version of a state-of-the-art algorithm for tracking human movement, DeeperCut 17 and can be used in a broad range of study systems with near human-level accuracy 18,19 .…”

OpenMonkeyChallenge: Dataset and Benchmark Challenges for Pose Tracking of Non-human Primates

“…To evaluate the generalizability of our dataset, we conduct a cross-dataset evaluation with OpenMonkeyPose (6) and MacaquePose (11). OpenMonkeyPose (6) consists of 195,228 annotated images simultaneously captured by 62 precisely arranged high-resolution video cameras.…”

“…(2) Data innovation: the tracking models learn the visual representation from a large annotated dataset that specifies the locations of landmarks. Existing publicly available datasets including OpenMonkeyPose (200K multiview macaque images in a specialized laboratory environment) (6) and Macaque-Pose (13K in-the-wild macaque images) (11) are important resources for the development of tracking algorithms, and as such, extend the boundary of pose tracking performance of NHPs. However, due to limited data diversity (appearance, pose, viewpoint, environment, and species), existing datasets are currently insufficient for learning generalizable tracking models.…”

Multiple Monkey Pose Estimation Using OpenPose

“…It has been shown that a deep learning approach is feasible [5][6] [7]. Although monkey pose could be estimated accurately, these works are limited to a single subject.…”