Deciphering choreographies of elaborate courtship displays of golden‐collared manakins using markerless motion capture

Janisch, Judith; Perinot, Elisa; Fusani, Leonida; Quigley, Cliodhna

doi:10.1111/eth.13161

Cited by 8 publications

(4 citation statements)

References 50 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For similar reasons, the proposed approach will not support annotation for flying birds or birds that change the shape of body parts while performing certain behaviors e.g. courtship [19].…”

Section: Limitations and Future Workmentioning

confidence: 99%

3D-POP -- An automated annotation approach to facilitate markerless 2D-3D tracking of freely moving birds with marker-based motion capture

Naik¹,

Chan²,

Junran³

et al. 2023

Preprint

View full text Add to dashboard Cite

Recent advances in machine learning and computer vision are revolutionizing the field of animal behavior by enabling researchers to track the poses and locations of freely moving animals without any marker attachment. However, large datasets of annotated images of animals for markerless pose tracking, especially high-resolution images taken from multiple angles with accurate 3D annotations, are still scant. Here, we propose a method that uses a motion capture (mo-cap) system to obtain a large amount of annotated data on animal movement and posture (2D and 3D) in a semi-automatic manner. Our method is novel in that it extracts the 3D positions of morphological keypoints (e.g eyes, beak, tail) in reference to the positions of markers attached to the animals. Using this method, we obtained, and offer here, a new dataset -3D-POP with approximately 300k annotated frames (4 million instances) in the form of videos having groups of one to ten freely moving birds from 4 different camera views in a 3.6m x 4.2m area. 3D-POP is the first dataset of flocking birds with accurate keypoint annotations in 2D and 3D along with bounding box and individual identities and will facilitate the development of solutions for problems of 2D to 3D markerless pose, trajectory tracking, and identification in birds.

show abstract

Section: Limitations and Future Workmentioning

confidence: 99%

3D-POP -- An automated annotation approach to facilitate markerless 2D-3D tracking of freely moving birds with marker-based motion capture

Naik¹,

Chan²,

Junran³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Satterfield et al, 2023; Vivancos & Closs, 2015), behaviours and interactions (e.g. Ballesta et al, 2014; Janisch et al, 2021), and physiology (e.g. Schiettekatte et al, 2022) can be inferred.…”

Section: Introductionmentioning

confidence: 99%

“…While capturing long‐term 3D video of animals and their behaviours with a high‐resolution and high‐sampling frequency is more straightforward in terrestrial and laboratory settings (e.g. Ballesta et al, 2014; Janisch et al, 2021; Karashchuk et al, 2021; Letessier et al, 2013; Wang et al, 2013; Zhu & Weng, 2007), capturing equivalent data for aquatic field studies is more challenging, particularly where constant power cannot be supplied. Underwater stereo‐video cameras have been used to quantify fish biomass (e.g.…”

Section: Introductionmentioning

confidence: 99%

A low‐cost, long‐running, open‐source stereo camera for tracking aquatic species and their behaviours

Dunkley,

Drewnicki

et al. 2023

Methods Ecol Evol

View full text Add to dashboard Cite

Ecologists are now widely utilising video data to quantify the behaviours and interactions of animals in the wild. This process can be facilitated by collecting videos in stereo, which can provide information about animals' positions, movements and behaviours in three‐dimensions (3D). However, there are no published designs that can collect underwater 3D stereo data at high spatial and temporal resolutions for extended periods (days). Here, we present complete hardware and software solutions for a long‐running, open‐source, underwater stereo camera rig, costing £1337. This stereo camera can continuously record aquatic species and their behaviours/interactions in high resolution (1080 p and 30 fps) and in 3D, over multiple days. We provide full design guides for the cameras and a travel‐friendly rig, and include guidance and open‐source code for calibrating the cameras in space and time. We also show how these cameras could be used to track animals' body parts and positions, and how their size, posture and behaviour can be inferred. This stereo camera will facilitate the collection of high‐resolution ecological and behavioural data, such as affiliative, agonistic or trophic interactions between species, which can inform us about the health and structure of ecosystems. These data will assist ecologists and conservationists in monitoring and understanding the impacts of current environmental pressures on ecosystem functioning.

show abstract

“…For this purpose, we combined motion tracking with a deep learning algorithm in order to achieve a high throughput unbiased approach. A convolutional neuronal network ("loopy") enabled us to acquire versatile and quantifiable data sets from video motion tracking and, in combination with scripting languages (Kil et al, 2020;Janisch et al, 2021;Mitoyen et al, 2021), to identify specialized behavioral phenotypes on a larger scale.…”

Section: Introductionmentioning

confidence: 99%

Characterization of tmt-opsin2 in Medaka Fish Provides Insight Into the Interplay of Light and Temperature for Behavioral Regulation

2021

View full text Add to dashboard Cite

One of the big challenges in the study of animal behavior is to combine molecular-level questions of functional genetics with meaningful combinations of environmental stimuli. Light and temperature are important external cues, influencing the behaviors of organisms. Thus, understanding the combined effect of light and temperature changes on wild-type vs. genetically modified animals is a first step to understand the role of individual genes in the ability of animals to cope with changing environments. Many behavioral traits can be extrapolated from behavioral tests performed from automated motion tracking combined with machine learning. Acquired datasets, typically complex and large, can be challenging for subsequent quantitative analyses. In this study, we investigate medaka behavior of tmt-opsin2 mutants vs. corresponding wild-types under different light and temperature conditions using automated tracking combined with a convolutional neuronal network and a Hidden Markov model-based approach. The temperatures in this study can occur in summer vs. late spring/early autumn in the natural habitat of medaka fish. Under summer-like temperature, tmt-opsin2 mutants did not exhibit changes in overall locomotion, consistent with previous observations. However, detailed analyses of fish position revealed that the tmt-opsin2 mutants spent more time in central locations of the dish, possibly because of decreased anxiety. Furthermore, a clear difference in location and overall movement was obvious between the mutant and wild-types under colder conditions. These data indicate a role of tmt-opsin2 in behavioral adjustment, at least in part possibly depending on the season.

show abstract

Deciphering choreographies of elaborate courtship displays of golden‐collared manakins using markerless motion capture

Abstract: This is an open access article under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in any medium, provided the original work is properly cited.

Cited by 8 publications

References 50 publications

3D-POP -- An automated annotation approach to facilitate markerless 2D-3D tracking of freely moving birds with marker-based motion capture

3D-POP -- An automated annotation approach to facilitate markerless 2D-3D tracking of freely moving birds with marker-based motion capture

A low‐cost, long‐running, open‐source stereo camera for tracking aquatic species and their behaviours

Characterization of tmt-opsin2 in Medaka Fish Provides Insight Into the Interplay of Light and Temperature for Behavioral Regulation

Contact Info

Product

Resources

About