Markerless tracking of an entire honey bee colony

Bożek, Katarzyna; Hebert, Laetitia; Portugal, Yoann; Mikheyev, Alexander S.; Stephens, Greg J.

doi:10.1038/s41467-021-21769-1

Cited by 37 publications

(20 citation statements)

References 87 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Despite its potentialities for measurements in dynamically evolving systems, entropy estimation via Bayesian inference is still a rather uncommon approach. Previous stud-ies have dealt with high-quality background treatment [55] and tracking of unlabeled organisms [31,56], resulting in precise measurements. However, differently from our efforts, the other algorithms have the disadvantages of needing a low-noise image and lacking the ability to extract dynamical features of the system, as their identification is made by a convolutional neural network (a black-box model).…”

Section: Discussionmentioning

confidence: 99%

“…The primary challenge of the multi-target tracking algorithm is the non-swapping of the labels when two objects (for example, animals being tracked) intersect. All of these multi-target video tracking challenges occur within natural colonies of social insects such as honey bees, Apis mellifera, as honey bee colonies are densely composed of individuals that exhibit similarities in their form and locomotion [31]. The quantitative understanding of worker behaviors within the honey bee colonies (e.g., swarm entropy [32], kinetic energy [33,34], spatial distribution, and collective activities [35]), however, is useful to recognize colony health, which can be used as indirect evidence of potential contamination by environmental pollutants.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Bayesian Multi-Targets Strategy to Track Apis mellifera Movements at Colony Level

Oliveira

Santos

Jumbo

et al. 2022

Insects

View full text Add to dashboard Cite

Interactive movements of bees facilitate the division and organization of collective tasks, notably when they need to face internal or external environmental challenges. Here, we present a Bayesian and computational approach to track the movement of several honey bee, Apis mellifera, workers at colony level. We applied algorithms that combined tracking and Kernel Density Estimation (KDE), allowing measurements of entropy and Probability Distribution Function (PDF) of the motion of tracked organisms. We placed approximately 200 recently emerged and labeled bees inside an experimental colony, which consists of a mated queen, approximately 1000 bees, and a naturally occurring beehive background. Before release, labeled bees were fed for one hour with uncontaminated diets or diets containing a commercial mixture of synthetic fungicides (thiophanate-methyl and chlorothalonil). The colonies were filmed (12 min) at the 1st hour, 5th and 10th days after the bees’ release. Our results revealed that the algorithm tracked the labeled bees with great accuracy. Pesticide-contaminated colonies showed anticipated collective activities in peripheral hive areas, far from the brood area, and exhibited reduced swarm entropy and energy values when compared to uncontaminated colonies. Collectively, our approach opens novel possibilities to quantify and predict potential alterations mediated by pollutants (e.g., pesticides) at the bee colony-level.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Bayesian Multi-Targets Strategy to Track Apis mellifera Movements at Colony Level

Oliveira

Santos

Jumbo

et al. 2022

Insects

View full text Add to dashboard Cite

show abstract

“…For example, human genomics have linked several genes to autism [2][3][4] , but we still know little about how these genetic changes increase the risk of autism. A 'computational ethology' 76 approach to social behavior analysis based on automatic posture tracking (such as pioneered in laboratory studies of insects, worms and fish 20,[77][78][79][80][81][82] and in field ethology [83][84][85][86] ) does not require us to a priori imagine how, e.g., autism-related gene perturbations manifest in mice, and can identify subtle changes in higher-order behavioral statistics without human observer bias. By recording days of social interactions, it may be possible to use methods from computational topology to ask how the high-dimensional space defined by touch, posture and movement dynamics is impacted by different genotypes or pathological conditions.…”

Section: Automatic Mapping Of Social Phenotypesmentioning

confidence: 99%

Automatic mapping of multiplexed social receptive fields by deep learning and GPU-accelerated 3D videography

Ebbesen

Froemke

2021

Preprint

View full text Add to dashboard Cite

Social interactions powerfully impact the brain and the body, but high-resolution descriptions of these important physical interactions are lacking. Currently, most studies rely on labor-intensive methods such as manual annotation. Scalable and objective tracking methods are required to understand the neural circuits underlying social behavior. Here we describe a hardware/software system and analysis pipeline that combines 3D videography, deep learning, physical modeling, and GPU-accelerated robust optimization, with automatic analysis of neuronal receptive fields recorded in interacting mice. Our system is capable of fully automatic multi-animal tracking with minimal errors (including in complete darkness) during complex, spontaneous social encounters, together with simultaneous electrophysiological recordings. We capture posture dynamics of multiple unmarked mice with high spatiotemporal precision (~2 mm, 60 frames/s). A generative model revealed the multiplexed ‘social receptive field’ of neurons in barrel cortex. This approach could be broadly useful for neurobehavioral studies of multiple animals interacting in complex low-light environments.

show abstract

Section: Automatic Mapping Of Social Phenotypesmentioning

confidence: 99%

Automatic mapping of multiplexed social receptive fields by deep learning and GPU-accelerated 3D videography

Ebbesen

Froemke

2020

Preprint

View full text Add to dashboard Cite

13Social interactions powerfully impact both the brain and the body, but high-resolution descriptions of these 14 important physical interactions are lacking. Currently, most studies of social behavior rely on labor-intensive 15 methods such as manual annotation of individual video frames. These methods are susceptible to experimenter 16 bias and have limited throughput. To understand the neural circuits underlying social behavior, scalable and 17 objective tracking methods are needed. We present a hardware/software system that combines 3D videography, 18 deep learning, physical modeling and GPU-accelerated robust optimization. Our system is capable of fully 19 automatic multi-animal tracking during naturalistic social interactions and allows for simultaneous electro-20 physiological recordings. We capture the posture dynamics of multiple unmarked mice with high spatial (~2 21 mm) and temporal precision (60 frames/s). This method is based on inexpensive consumer cameras and is 22 implemented in python, making our method cheap and straightforward to adopt and customize for studies of 23 neurobiology and animal behavior. 24 RESULTS 67 Raw data acquisition 68We established an experimental setup that allowed us to capture synchronized color images and depth images 69 from multiple angles, while simultaneously recording synchronized neural data ( Fig. 1a). We used inexpen-70 sive, state-of-the-art 'depth cameras' for computer vision and robotics. These cameras contain several imaging 71 modules: one color sensor, two infrared sensors and an infrared laser projector ( Fig. 1b). Imaging data pipe-72 lines, as well as intrinsic and extrinsic sensor calibration parameters can be accessed over USB through a 73 C/C++ SDK with Python bindings. We placed four depth cameras, as well as four synchronization LEDs 74 around a transparent acrylic cylinder which served as our behavioral arena ( Fig. 1c). 75 76 Each depth camera projects a static dot pattern across the imaged scene, adding texture in the infrared spec-77 trum to reflective surfaces ( Fig. 1d). By imaging this highly-textured surface simultaneously with two infrared 78 sensors per depth camera, it is possible to estimate the distance of each pixel in the infrared image to the depth 79 camera by stereopsis (by locally estimating the binocular disparity between the textured images). Since the 80 dot pattern is static and only serves to add texture, multiple cameras do not interfere with each other and it is 81 possible to image the same scene from multiple angles. This is one key aspect of our method, not possible 82 with depth imaging systems that rely on actively modulated light (such as the Microsoft Kinect system and 83 earlier versions of the Intel Realsense cameras). 84 85Since mouse movement is fast 13 , it is vital to minimize motion blur in the infrared images and thus the final 86 3D data ('point-cloud'). To this end, our method relies on two key features. First, we use depth cameras where 87 the infrared sensors have a global shutter (e.g., Intel D435) rathe...

show abstract

Markerless tracking of an entire honey bee colony

Cited by 37 publications

References 87 publications

Bayesian Multi-Targets Strategy to Track Apis mellifera Movements at Colony Level

Bayesian Multi-Targets Strategy to Track Apis mellifera Movements at Colony Level

Automatic mapping of multiplexed social receptive fields by deep learning and GPU-accelerated 3D videography

Automatic mapping of multiplexed social receptive fields by deep learning and GPU-accelerated 3D videography

Contact Info

Product

Resources

About