Joining the dots: reconstructing 3D environments and movement paths using animal-borne devices

Abstract

Background: Animal-attached sensors are increasingly used to provide insights on behaviour and physiology. However, such tags usually lack information on the structure of the surrounding environment from the perspective of a study animal and thus may be unable to identify potentially important drivers of behaviour. Recent advances in robotics and computer vision have led to the availability of integrated depth-sensing and motion-tracking mobile devices. These enable the construction of detailed 3D models of an environment within which motion can be tracked without reliance on GPS. The potential of such techniques has yet to be explored in the field of animal biotelemetry. This report trials an animal-attached structured light depth-sensing and visual-inertial odometry motion-tracking device in an outdoor environment (coniferous forest) using the domestic dog (Canis familiaris) as a compliant test species.Results: A 3D model of the forest environment surrounding the subject animal was successfully constructed using point clouds. The forest floor was labelled using a progressive morphological filter. Trees trunks were modelled as cylinders and identified by random sample consensus. The predicted and actual presence of trees matched closely, with an object-level accuracy of 93.3%. Individual points were labelled as belonging to tree trunks with a precision, recall, and F β score of 1.00, 0.88, and 0.93, respectively. In addition, ground-truth tree trunk radius measurements were not significantly different from random sample consensus model coefficient-derived values. A first-person view of the 3D model was created, illustrating the coupling of both animal movement and environment reconstruction. Conclusions:Using data collected from an animal-borne device, the present study demonstrates how terrain and objects (in this case, tree trunks) surrounding a subject can be identified by model segmentation. The device pose (position and orientation) also enabled recreation of the animal's movement path within the 3D model. Although some challenges such as device form factor, validation in a wider range of environments, and direct sunlight interference remain before routine field deployment can take place, animal-borne depth sensing and visual-inertial odometry have great potential as visual biologging techniques to provide new insights on how terrestrial animals interact with their environments.

Life in 2.5D: Animal Movement in the Trees

“…Technological advances allow for increasingly accurate and detailed mapping of complex canopy habitat structure. Light Detection and Ranging (LiDAR) systems can map the threedimensional architecture of forests (Davies and Asner, 2014;Rodríguez-Ronderos et al, 2016;McClune, 2018;Moorthy et al, 2019), and when mounted on a drone or a plane, offer promising opportunities for balancing fine-scale and landscapelevel measurements of tree and forest structure, canopy height, and vegetation density (Asner et al, 2008). Satellite-borne LiDAR systems can also measure vertical canopy structure at larger spatial scales, although with coarser resolution (e.g., Global Ecosystem Dynamics Investigation, GEDI) (Dubayah et al, 2020;Lang et al, 2021).…”

Through Hawks’ Eyes: Synthetically Reconstructing the Visual Field of a Bird in Flight

“…In both of these studies, the environments mapped were , but there is currently growing interest in reconstructing an animal’s experience of its environment at much larger scales, relevant to ecology and conservation (Tuia et al, 2022 ). This interest has led to demonstrations of animal-borne 3D mapping sensors (McClune, 2018 ), and a mobile-camera method for embedding an animal’s track in an aerial view of its environment (Haalck et al, 2020 ).…”

Dead-reckoning animal movements in R: a reappraisal using Gundog.Tracks

“…Reconstructing animal movement paths is an important tool in ecology, providing insights into animal space-use, behaviour and habitat selection [1][2][3]. However, accurate estimation of paths at fine temporal scales has proved a persistent challenge [4,5].…”