idtracker.ai: Tracking all individuals in large collectives of unmarked animals

Romero-Ferrero, Francisco; Bergomi, Mattia G.; Hinz, Robert S.; Heras, Francisco J. H.; Polavieja, Gonzalo G. de

doi:10.48550/arxiv.1803.04351

Cited by 3 publications

(5 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, performing such an experiment with a large group of mutant fish is challenging, as mutants display a relatively high mortality rate. In addition, tracking larger groups requires better camera systems (see [65] for example), more optimised algorithms (see [66,67] for examples), and a very large space to avoid the interruption from the boundary. These requirements are beyond our current experimental capability.…”

Section: Discussionmentioning

confidence: 99%

Tuning Collective Behaviour in Zebrafish with Genetic Modification

Yang,

Kawafi,

Tong

et al. 2024

Preprint

View full text Add to dashboard Cite

Zebrafish collective behaviour is widely used to assess their physical and mental state, serving as a valuable tool to assess the impact of ageing, disease genetics, and the effect of drugs. The essence of these macroscopic phenomena can be represented by active matter models, where the individuals are abstracted as interactive self-propelling agents. The behaviour of these agents depends on a set of parameters in a manner reminiscent of those between the constituents of physical systems. In a few cases, the system may be controlled at the level of the individual constituents such as the interactions between colloidal particles, or the enzymatic behaviour of de novo proteins. Usually, however, while the collective behaviour may be influenced by environmental factors, it typically cannot be changed at will. Here, we challenge this scenario in a biological context by genetically modifying zebrafish. We thus demonstrate the potential of genetic modification in the context of controlling the collective behaviour of biological active matter systems at the level of the constituents, rather than externally. In particular, we probe the effect of the lack of col11a2 gene in zebrafish, which causes the early onset of osteoarthritis. The resulting col11a2 -/- zebrafish exhibited compromised vertebral column properties, bent their body less while swimming, and took longer to change their orientations. Surprisingly, a group of 25 mutant fish exhibited more orderly collective motion than the wildtype. We show that the collective behaviour of wildtype and col11a2 zebrafish are captured with a simple active matter model, in which the mutant fish are modelled by self–propelling agents with a higher orientational noise on average. In this way, we demonstrate the possibility of tuning a biological system, changing the state space it occupies when interpreted with a simple active matter model.

show abstract

Section: Discussionmentioning

confidence: 99%

Tuning Collective Behaviour in Zebrafish with Genetic Modification

Yang,

Kawafi,

Tong

et al. 2024

Preprint

View full text Add to dashboard Cite

show abstract

“…Compared to the MOT16 benchmark, honeybee colonies contain a significantly higher number of individuals (∼ 1, 000 each), a number prohibitively large for the comprehensive exploration of track association hypotheses. Individual bees are generally of the same size, largely similar, and not available for recording in isolation, which limits the usefulness of comprehensive appearance models [18]. The beehives also exhibit a varied, uneven background composed of comb cells, brood, and food, the viewpoint of the recordings is static, and object classes are limited to two types of bee body postures.…”

Section: Related Workmentioning

confidence: 99%

“…Due to the difficulties of multi-object tracking, studies in biological systems often produce collective recordings under controlled laboratory conditions or with the marking of individuals. Typical experimental conditions involve plain 2D or 3D arenas with uniform illumination and high foreground-background contrast [18,19]. The controlled aspect of these arrangements allows for accurate object detection as well as extraction of comprehensive appearance cues from large volumes of recordings of each individual [18].…”

Section: Related Workmentioning

confidence: 99%

“…Typical experimental conditions involve plain 2D or 3D arenas with uniform illumination and high foreground-background contrast [18,19]. The controlled aspect of these arrangements allows for accurate object detection as well as extraction of comprehensive appearance cues from large volumes of recordings of each individual [18]. The marking of individuals with unique barcodes has also been extensively used in the study of social insects [2,20,21], but has important feasibility limitations in large colonies with short generation time spans.…”

Section: Related Workmentioning

confidence: 99%

“…We captured cues to individual identity across varying visual appearances, an attribute we denote as pixel personality, by using the object recognition network Inception V3 [22]. We also note previous approaches that exploit subtle but consistent visual differences for multianimal tracking [18,23]. The initial track segments described above were used as a training set for learning object identities.…”

Section: Joining Track Fragments Using Appearance-based Matchingmentioning

confidence: 99%

See 2 more Smart Citations

Pixel personality for dense object tracking in a 2D honeybee hive

Bożek

Hebert

Mikheyev

et al. 2019

Preprint

View full text Add to dashboard Cite

Tracking large numbers of densely-arranged, interacting objects is challenging due to occlusions and the resulting complexity of possible trajectory combinations, as well as the sparsity of relevant, labeled datasets. Here we describe a novel technique of collective tracking in the model environment of a 2D honeybee hive in which sample colonies consist of N ∼ 10 3 highly similar individuals, tightly packed, and in rapid, irregular motion. Such a system offers universal challenges for multiobject tracking, while being conveniently accessible for image recording. We first apply an accurate, segmentation-based object detection method to build initial short trajectory segments by matching object configurations based on class, position and orientation. We then join these tracks into full single object trajectories by creating an object recognition model which is adaptively trained to recognize honeybee individuals through their visual appearance across multiple frames, an attribute we denote as pixel personality. Overall, we reconstruct ∼ 46% of the trajectories in 5 min recordings from two different hives and over 71% of the tracks for at least 2 min. We provide validated trajectories spanning 3, 000 video frames of 876 unmarked moving bees in two distinct colonies in different locations and filmed with different pixel resolutions, which we expect to be useful in the further development of general-purpose tracking solutions.

show abstract