Application of a novel deep learning–based 3D videography workflow to bat flight

Abstract

Studying the detailed biomechanics of flying animals requires accurate three‐dimensional coordinates for key anatomical landmarks. Traditionally, this relies on manually digitizing animal videos, a labor‐intensive task that scales poorly with increasing framerates and numbers of cameras. Here, we present a workflow that combines deep learning–powered automatic digitization with filtering and correction of mislabeled points using quality metrics from deep learning and 3D reconstruction. We tested our workflow using a particularly challenging scenario: bat flight. First, we documented four bats flying steadily in a 2 m3 wind tunnel test section. Wing kinematic parameters resulting from manually digitizing bats with markers applied to anatomical landmarks were not significantly different from those resulting from applying our workflow to the same bats without markers for five out of six parameters. Second, we compared coordinates from manual digitization against those yielded via our workflow for bats flying freely in a 344 m3 enclosure. Average distance between coordinates from our workflow and those from manual digitization was less than a millimeter larger than the average human‐to‐human coordinate distance. The improved efficiency of our workflow has the potential to increase the scalability of studies on animal flight biomechanics.

Three-dimensional body reconstruction enables quantification of liquid consumption in small invertebrates

“…In our case, this was mitigated by a high sample size. However, if possible, building a recording system with more cameras would mitigate the need for the animals to be in specific positions, enabling for example the tracking of animals mid-flight (Maya et al ., 2023; Håkansson et al ., 2024). This could overcome limitations such as occlusions, perspective distortions or ambiguity in depth estimation.…”

“…In most cases, a basic recording configuration, consisting of two perpendicular cameras, capturing a top-view and side-view of the features of interest, should suffice. Nevertheless, additional cameras, placed at varying angles may be required, particularly if recording subjects mid-flight (Maya et al ., 2023; Håkansson et al ., 2024), for example. Regardless of the number of cameras used, it is imperative these are stereo-configured and time-synchronized to ensure overlapping views and simultaneous visibility of features of interest across multiple cameras, allowing for the feature to be reconstructed in a three-dimensional space.…”