Interactive rodent behavior annotation in video using active learning

Abstract

Manual annotation of rodent behaviors in video is time-consuming. By learning a classifier, we can automate the labeling process. Still, this strategy requires a sufficient number of labeled examples. Moreover, we need to train new classifiers when there is a change in the set of behaviors that we consider or in the manifestation of these behaviors in video. Consequently, there is a need for an efficient way to annotate rodent behaviors. In this paper we introduce a framework for interactive behavior annotation in video based on active learning. By putting a human in the loop, we alternate between learning and labeling. We apply the framework to three rodent behavior datasets and show that we can train accurate behavior classifiers with a strongly reduced number of labeled samples. We confirm the efficacy of the tool in a user study demonstrating that interactive annotation facilitates efficient, high-quality behavior measurements in practice.

Deep learning-based system for real-time behavior recognition and closed-loop control of behavioral mazes using depth sensing

“…Besides reducing human bias and subjectivity, and consequently allowing for the standardization of measurements across laboratories, behavioral patterns that were once unnoticed to a human observer may now be explored at different scales and resolutions [4][5][6] . The first approaches to successfully combine computer vision and machine learning techniques typically relied on hand-crafted features extracted from images or video sequences that can be then used for automated behavior classification using supervised [7][8][9][10] or unsupervised [11][12][13] learning methods. However, such approaches are highly dependent on domain expertise for feature engineering, often losing their generalization capability in the presence of a new environment/scenario.…”

Action Recognition Using a Spatial-Temporal Network for Wild Felines

“…With the rise in computer vision and pattern recognition technology, impressive progress has been made in image classification [ 12 ] and object detection [ 13 ], motivating researchers to apply artificial intelligence for action recognition of animals, such as mice [ 14 , 15 , 16 , 17 ], domestic animals (cows and pigs) [ 18 , 19 , 20 , 21 ], Tibetan antelope [ 22 ], and ants [ 23 ]. However, some of these methods limit the research subjects to small animals in the laboratory [ 14 , 15 , 16 , 17 ] while other methods usually make simple judgments about abnormal behaviors of animals [ 18 , 19 , 20 , 21 , 22 ]. Research on wild felines is extremely scarce.…”