Integrating real-time data analysis into automatic tracking of social insects

Abstract

Automatic video tracking has become a standard tool for investigating the social behaviour of insects. The recent integration of computer vision in tracking technologies will probably lead to fully automated behavioural pattern classification within the next few years. However, many current systems rely on offline data analysis and use computationally expensive techniques to track pre-recorded videos. To address this gap, we developed BACH (Behaviour Analysis maCHine), a software that performs video tracking of insect groups in real time. BACH uses object recognition via convolutional neural networks and identifies individually tagged insects via an existing matrix code recognition algorithm. We compared the tracking performances of BACH and a human observer (HO) across a series of short videos of ants moving in a two-dimensional arena. We found that BACH detected ant shapes only slightly worse than the HO. However, its matrix code-mediated identification of individual ants only attained human-comparable levels when ants moved relatively slowly, and fell when ants walked relatively fast. This happened because BACH had a relatively low efficiency in detecting matrix codes in blurry images of ants walking at high speeds. BACH needs to undergo hardware and software adjustments to overcome its present limits. Nevertheless, our study emphasizes the possibility of, and the need for, further integrating real-time data analysis into the study of animal behaviour. This will accelerate data generation, visualization and sharing, opening possibilities for conducting fully remote collaborative experiments.

Dual‐fluorescence imaging and automated trophallaxis detection for studying multi‐nutrient regulation in superorganisms

“…Insect size is typically not an obstacle as extremely small tags are available (Stroeymeyt et al., 2018). Alternatively, there are methods to track untagged insects (Romero‐Ferrero et al., 2019; Sclocco et al., 2021). The choice of lenses and camera resolution may depend on insect size and experimental requirements.…”

“…395 Ferrero et al, 2019;Sclocco, Ong, et al, 2021). The choice of lenses and camera resolution may depend on 396 insect size and experimental requirements.…”

NAPS: Integrating pose estimation and tag‐based tracking

“…Appearance‐based models require a significant time commitment for model training and struggle to distinguish visually similar individuals (Lauer et al, 2022; Pereira et al, 2022; Romero‐Ferrero et al, 2019). Other methods, such as TRex and Argos, have relied on identifying where individuals are in extremely close proximity (crossing) and using neural networks specifically to maintain identity across these crosses (Sclocco et al, 2021; Walter & Couzin, 2021).…”

Anttracker: A Low-Cost and Efficient Computer Vision Approach to Research Leaf-Cutter Ants Behavior

“…There exist several successful tools to track insects [29,30]. More recently, in [31] they propose a device, illumination system and object detectors based on CNN to identify ants in a clear background. In this case, ants are tracked with a codebar attached to their bodies.…”