FlyLimbTracker: an active contour based approach for leg segment tracking in unmarked, freely behavingDrosophila

Abstract

24Understanding the biological underpinnings of movement and action requires 25 the development of tools for precise, quantitative, and high-throughput 26 measurements of animal behavior. Drosophila melanogaster provides an ideal 27 model for developing such tools: the fly has unparalleled genetic accessibility 28 and depends on a relatively compact nervous system to generate 29 sophisticated limbed behaviors including walking, reaching, grooming, 30 courtship, and boxing. Here we describe a method that uses active contours 31 to semi-automatically track body and leg segments from video image 32 sequences of unmarked, freely behaving Drosophila. We show that this 33 approach is robust to wide variations in video spatial and temporal resolution 34 and that it can be used to measure leg segment motions during a variety of 35 locomotor and grooming behaviors. FlyLimbTracker, the software 36 implementation of this method, is open-source and our approach is 37 generalizable. This opens up the possibility of tracking leg movements in 38 other species by modifications of underlying active contour models. 39 40 41 42 43 Author Summary 44 In terrestrial animals, including humans, fundamental actions like locomotion 45 and grooming emerge from the displacement of multiple limbs through space. 46 Therefore, precise measurements of limb movements are critical for 47 investigating and, ultimately, understanding the neural basis for behavior. The 48 vinegar fly, Drosophila melanogaster, is an attractive animal model for 49 uncovering general principles about limb control since its genome and 50 nervous system are easy to manipulate. However, existing methods for 51 measuring leg movements in freely behaving Drosophila have significant 52 drawbacks: they require complicated experimental setups and provide limited 53 information about each leg. Here we report a new method -and provide its 54 open-source software implementation, FlyLimbTracker -for tracking the body 55 and leg segments of freely behaving flies using only computational image 56 processing approaches. We illustrate the power of this method by tracking fly 57 limbs during five distinct walking and grooming behaviors and from videos 58 across a wide range of spatial and temporal resolutions. Our approach is 59 generalizable, allowing researchers to use and customize our software for 60 limb tracking in Drosophila and in other species. 61 62 FlyLimbTracker is implemented in Java as a freely available plug-in for 131 Icy, a cross-platform, multi-purpose image processing environment [16]. 132 Briefly, FlyLimbTracker performs leg segment tracking in several steps. First, 133 the user is asked to manually initialize the position of a fly's body and leg 134 segments in a single frame of the image sequence. This information is 135 combined with image features to propagate body and leg segmentation to the 136

Fast animal pose estimation using deep neural networks

“…Previous attempts at automated body-part tracking in insects and mammals have 53 relied on either physically constraining the animal and having it walk on a spherical treadmill 17 54 or linear track 18 , applying physical markers to the animal 17,19 , or utilizing specialized equipment 55 such as depth cameras [20][21][22] , frustrated total internal reflection imaging 23,24 or multiple cameras 56 25 . Meanwhile, approaches designed to operate without constraining the natural space of 57 behaviors make use of image processing techniques that are sensitive to imaging conditions 58 and require manual correction even after full training 26 . 59…”

A cost effective high-resolution climbing assay applied to Drosophila Parkinson’s and proprioception mutants reveal novel behavioural phenotypes

“…Tools ranging from a method, very similar to the one presented here, developed almost a decade ago (19), to more recent machine learning based behaviour annotation tools are being developed extensively (20,21). Some of these tools, specific for fly limb tracking have greatly contributed to our understanding of fly locomotion at the level of individual leg control (22)(23)(24)(25). However, technical challenges limit our ability to quickly screen through large numbers and over long distances using these methods.…”

MARGO (Massively Automated Real-time GUI for Object-tracking), a platform for high-throughput ethology

“…Furthermore, the need to record high-quality, highresolution video data can make it challenging to track animals over long experiments. Some methods of postural segmentation require manual addition of limb markers 17 , splines fit in post-processing 18 , or computationally heavy machine vision in post-processing 3;4;8 . In all cases, the need to separate tracking and recording can be ratelimiting for experiments.…”