OpenMonkeyChallenge: Dataset and Benchmark Challenges for Pose Tracking of Non-human Primates

Yao, Yuan; Mohan, Abhiraj; Bliss‐Moreau, Eliza; Coleman, Kristine; Freeman, Sienna; Machado, Christopher J.; Raper, Jessica; Zimmermann, Jan; Hayden, Benjamin Y.; Park, Hyun Soo

doi:10.1101/2021.09.08.459549

Cited by 2 publications

(2 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Nonetheless, it still lags behind the accuracy of human pose estimation. Open-MonkeyChallenge is designed to address this limitation by creating a 6 fold-bigger-data set across multiple species to enable learning a shared representation between them (Yao et al, 2021).…”

Section: Target-specific Model Versus Generalizable Modelmentioning

confidence: 99%

“…The MacaquePose data set that is made of 17,000 annotated internet images (Labugen et al, 2020). It is an inspiration for the OpenMonkeyChallenge project (Yao et al, 2021). It includes a large variety of appearances, poses, and viewpoints of macaques in the wild, which can be combined with state‐of‐the‐art pose detection models such as DeepLabCut and LEAP.…”

Section: The Challenges Of Big Behavior In Primatesmentioning

confidence: 99%

See 1 more Smart Citation

Automated pose estimation in primates

Hayden

Park

Zimmermann

2021

American J Primatol

Self Cite

View full text Add to dashboard Cite

Understanding the behavior of primates is important for primatology, for psychology, and for biology more broadly. It is also important for biomedicine, where primates are an important model organism, and whose behavior is often an important variable of interest. Our ability to rigorously quantify behavior has, however, long been limited. On one hand, we can rigorously quantify low-information measures like preference, looking time, and reaction time; on the other, we can use more gestalt measures like behavioral categories tracked via ethogram, but at high cost and with high variability. Recent technological advances have led to a major revolution in behavioral measurement that offers affordable and scalable rigor. Specifically, digital video cameras and automated pose tracking software can provide measures of full-body position (i.e., pose) of primates over time (i.e., behavior) with high spatial and temporal resolution. Pose-tracking technology in turn can be used to infer behavioral states, such as eating, sleeping, and mating. We call this technological approach behavioral imaging. In this review, we situate the behavioral imaging revolution in the history of the study of behavior, argue for investment in and development of analytical and research techniques that can profit from the advent of the era of big behavior, and propose that primate centers and zoos will take on a more central role in relevant fields of research than they have in the past.

show abstract

Section: Target-specific Model Versus Generalizable Modelmentioning

confidence: 99%

Section: The Challenges Of Big Behavior In Primatesmentioning

confidence: 99%

Automated pose estimation in primates

Hayden

Park

Zimmermann

2021

American J Primatol

Self Cite

View full text Add to dashboard Cite

show abstract

Towards Automated Ethogramming: Cognitively-Inspired Event Segmentation for Streaming Wildlife Video Monitoring

Mounir¹,

Shahabaz²,

Gula³

et al. 2023

Int J Comput Vis

View full text Add to dashboard Cite

Advances in visual perceptual tasks have been mainly driven by the amount, and types, of annotations of large-scale datasets. Researchers have focused on fully-supervised settings to train models using offline epoch-based schemes. Despite the evident advancements, limitations and cost of manually annotated datasets have hindered further development for event perceptual tasks, such as detection and localization of objects and events in videos. The problem is more apparent in zoological applications due to the scarcity of annotations and length of videos-most videos are at most ten minutes long. Inspired by cognitive theories, we present a self-supervised perceptual prediction framework to tackle the problem of temporal event segmentation by building a stable representation of event-related objects. The approach is simple but effective. We rely on LSTM predictions of high-level features computed by a standard deep learning backbone. For spatial segmentation, the stable representation of the object is used by an attention mechanism to filter the input features before the prediction step. The self-learned attention maps effectively localize the object as a side effect of perceptual prediction. We demonstrate our approach on long videos from continuous wildlife video monitoring, spanning multiple days at 25 FPS. We aim to facilitate automated ethogramming by detecting and localizing events without the need for labels. Our approach is trained in an online manner on streaming input and requires only a single pass through the video, with no separate training set. Given the lack of long and realistic (includes real-world challenges) datasets, we introduce a new wildlife video dataset–nest monitoring of the Kagu (a flightless bird from New Caledonia)–to benchmark our approach. Our dataset features a video from 10 days (over 23 million frames) of continuous monitoring of the Kagu in its natural habitat. We annotate every frame with bounding boxes and event labels. Additionally, each frame is annotated with time-of-day and illumination conditions. We will make the dataset, which is the first of its kind, and the code available to the research community. We find that the approach significantly outperforms other self-supervised, traditional (e.g., Optical Flow, Background Subtraction) and NN-based (e.g., PA-DPC, DINO, iBOT), baselines and performs on par with supervised boundary detection approaches (i.e., PC). At a recall rate of 80%, our best performing model detects one false positive activity every 50 min of training. On average, we at least double the performance of self-supervised approaches for spatial segmentation. Additionally, we show that our approach is robust to various environmental conditions (e.g., moving shadows). We also benchmark the framework on other datasets (i.e., Kinetics-GEBD, TAPOS) from different domains to demonstrate its generalizability. The data and code are available on our project page: https://aix.eng.usf.edu/research_automated_ethogramming.html

show abstract

OpenMonkeyChallenge: Dataset and Benchmark Challenges for Pose Tracking of Non-human Primates

Cited by 2 publications

References 39 publications

Automated pose estimation in primates

Automated pose estimation in primates

Towards Automated Ethogramming: Cognitively-Inspired Event Segmentation for Streaming Wildlife Video Monitoring

Contact Info

Product

Resources

About