OpenMonkeyChallenge: Dataset and Benchmark Challenges for Pose Estimation of Non-human Primates

Yao, Yuan; Bala, Praneet C.; Mohan, Abhiraj; Bliss‐Moreau, Eliza; Coleman, Kristine; Freeman, Sienna; Machado, Christopher J.; Raper, Jessica; Zimmermann, Jan; Hayden, Benjamin Y.; Park, Hyun Soo

doi:10.1007/s11263-022-01698-2

Cited by 11 publications

(35 citation statements)

References 52 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We first trained each model on the MacaquePose dataset 76 , and then supplemented training with annotated images from our dataset. We trained the monkey detection Faster R-CNN model 77 , and then the pose estimation model HRNet-W48 78 , which has been shown to perform well in tracking myriad primate species 79 .…”

Section: Behavioral Quantification: a Monkey Detection And Pose Estim...mentioning

confidence: 99%

Neural signatures of natural behavior in socializing macaques

Testard

Tremblay

Parodi

et al. 2023

Preprint

View full text Add to dashboard Cite

Our understanding of the neurobiology of primate behavior largely derives from artificial tasks in highly-controlled laboratory settings, overlooking most natural behaviors primate brains evolved to produce1. In particular, how primates navigate the multidimensional social relationships that structure daily life and shape survival and reproductive success remains largely unexplored at the single neuron level. Here, we combine ethological analysis with new wireless recording technologies to uncover neural signatures of natural behavior in unrestrained, socially interacting pairs of rhesus macaques within a larger colony. Population decoding of single neuron activity in prefrontal and temporal cortex unveiled robust encoding of 24 species-typical behaviors, which was strongly modulated by the presence and identity of surrounding monkeys. Male-female partners demonstrated near-perfect reciprocity in grooming, a key behavioral mechanism supporting friendships and alliances, and neural activity maintained a running account of these social investments. When confronted with an aggressive intruder, behavioral and neural population responses reflected empathy and were buffered by the presence of a partner. Surprisingly, neural signatures in prefrontal and temporal cortex were largely indistinguishable and irreducible to visual and motor contingencies. By employing an ethological approach to the study of primate neurobiology, we reveal a highly-distributed neurophysiological record of social dynamics, a potential computational foundation supporting communal life in primate societies, including our own.

show abstract

Section: Behavioral Quantification: a Monkey Detection And Pose Estim...mentioning

confidence: 99%

Neural signatures of natural behavior in socializing macaques

Testard

Tremblay

Parodi

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Other notable datasets contain images instead of videos and focus on capturing variations in terms of specific taxa e.g. mammals [48], birds [41] and monkeys [47], or specific species e.g. zebras [14], all of these focus on solving problems for a single individual recorded from a single viewpoint.…”

Section: D Posturementioning

confidence: 99%

“…Although animals have been included in many popular image datasets collected from the internet such as ImageNet [9] and COCO [26], those datasets have not fulfilled more specific needs of animal behavior researchers. Hence, recently several datasets have been created with a focus on animal behavior research, such as species classification [12,37,40,41,48], behavioral classification [27,31,48] and posture tracking [2,10,15,24,47].…”

Section: Introductionmentioning

confidence: 99%

3D-POP -- An automated annotation approach to facilitate markerless 2D-3D tracking of freely moving birds with marker-based motion capture

Naik¹,

Chan²,

Junran³

et al. 2023

Preprint

View full text Add to dashboard Cite

Recent advances in machine learning and computer vision are revolutionizing the field of animal behavior by enabling researchers to track the poses and locations of freely moving animals without any marker attachment. However, large datasets of annotated images of animals for markerless pose tracking, especially high-resolution images taken from multiple angles with accurate 3D annotations, are still scant. Here, we propose a method that uses a motion capture (mo-cap) system to obtain a large amount of annotated data on animal movement and posture (2D and 3D) in a semi-automatic manner. Our method is novel in that it extracts the 3D positions of morphological keypoints (e.g eyes, beak, tail) in reference to the positions of markers attached to the animals. Using this method, we obtained, and offer here, a new dataset -3D-POP with approximately 300k annotated frames (4 million instances) in the form of videos having groups of one to ten freely moving birds from 4 different camera views in a 3.6m x 4.2m area. 3D-POP is the first dataset of flocking birds with accurate keypoint annotations in 2D and 3D along with bounding box and individual identities and will facilitate the development of solutions for problems of 2D to 3D markerless pose, trajectory tracking, and identification in birds.

show abstract

“…The application of this methodological innovation to the neuroscience research field is now rapidly expanding 11,[13][14][15][16][17] , as it is expected to have a potential to bring about fundamental changes in how to design behavioral experiments on NHPs which have long been carried out in a head-fixed condition. In the past decades, accumulated evidence from a number of research works, such as ethological studies on wild animals, suggests the continuity of behavioral repertoires across primates including humans [19][20][21] .…”

Section: Introductionmentioning

confidence: 99%

“…Two issues should be solved to achieve a methodological improvement in designing behavioral experiments on marmosets. First, the practical use of "deep neural networks" for behavioral analysis demands both a huge volume of ground truth data 14,16 and an analytic pipeline that reconstructs 3D poses of multiple animals simultaneously while recognizing individuals. Second, even if the best effort is made to establish such a system, a major question still arises as to how effective this approach is to evaluate natural behaviors of freely moving marmosets.…”

Section: Introductionmentioning

confidence: 99%

Establishing an AI-based evaluation system that quantifies social/pathophysiological behaviors of common marmosets

Kaneko,

Matsumoto,

et al. 2023

Preprint

View full text Add to dashboard Cite

Nonhuman primates (NHPs) are indispensable animal models by virtue of the continuity of behavioral repertoires across primates, including humans. However, assessment of behavioral actions at the laboratory level has so far been limited. By applying multiple deep-neural networks trained with large-scale datasets, we established an evaluation system that could reconstruct and estimate 3D poses of common marmosets, a small NHP that is suitable for analyzing complex natural behaviors in laboratory setups. We further developed downstream analytic methodologies to quantify a variety of behavioral parameters beyond simple motion kinematics, such as social interactions and internal state behind actions, obtained solely from 3D pose data. Moreover, a fully unsupervised analysis enabled us to detect progressively-appearing symptomatic behaviors over a year in a Parkinson's disease model. The high-throughput and versatile nature of our analytic pipeline will open a new avenue for various neuroscience fields dealing with big-data analyses of social/pathophysiological behaviors in NHPs.

show abstract

OpenMonkeyChallenge: Dataset and Benchmark Challenges for Pose Estimation of Non-human Primates

Cited by 11 publications

References 52 publications

Neural signatures of natural behavior in socializing macaques

Neural signatures of natural behavior in socializing macaques

3D-POP -- An automated annotation approach to facilitate markerless 2D-3D tracking of freely moving birds with marker-based motion capture

Establishing an AI-based evaluation system that quantifies social/pathophysiological behaviors of common marmosets

Contact Info

Product

Resources

About