Through hawks’ eyes: reconstructing a bird’s visual field in flight to study gaze strategy and attention during perching and obstacle avoidance

Miñano, Sofía; Gk, Taylor

doi:10.1101/2021.06.16.446415

Cited by 4 publications

(1 citation statement)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Even though the MS-based adaptive interaction paves a promising route to migrate the bionic mechanisms to bio-inspired swarm robotics, we openly admit that it still requires to pay for great efforts, such as precisely and eidetically reconstructing the visual fields of individual perception 30,62,63 , bridging the gap between mathematical modeling of motion salience and perceptual devices on robot 64,65 , etc. On the other hand, recently, neuroscience has made tremendous progress in understanding various aspects of relations between sensory signals and visuomotor stream in larval zebrafish 66,67 .…”

Section: Discussionmentioning

confidence: 99%

Perception of Motion Salience Shapes the Emergence of Collective Motions

Xiao,

Lei,

Zheng

et al. 2023

Preprint

View full text Add to dashboard Cite

As one of the most common and spectacular manifestations of coordinated behavior, collective motion is the spontaneous emergence of the ordered movement in a system consisting of many self-propelled agents, e.g., flocks of birds, schools of fish, herds of animals, and human crowds. Despite extensive studies on collective motions, a systems-level understanding of different motion patterns of collective behaviors is still lacking. This further hinders the adoption of bio-inspired mechanisms for applications of swarm robotics. Here, by leveraging three large bird-flocking datasets, we systematically investigate the emergence of different patterns of collective motions:mobbing, circling, andtransit. We find that flocks with higher maneuverable motions (i.e.,mobbingandcircling) prefer to evolve a more nested structure of leader-follower (LF) relations and a clear hierarchy to mitigate the damage of individual freedom to group cohesion. In contrast, flocks with smooth motion (i.e.,transit) do not display this tactful strategy to organize the group. To explain this empirical finding, we propose a measure based on the perception of motion salience (MS) to quantify the trade-off between individual freedom and group cohesion. Moreover, we perform the correlation analysis between LF and MS, finding that individuals with higher MS tend to lead the group with higher maneuverable motions. Those findings prompt us to develop a swarm model with adaptive MS-based (AMS) interactions and confirm that AMS interactions are responsible for the emergence of nested and hierarchical LF relations in the flocks with highly maneuverable motions. Finally, we implement AMS interactions in swarm robotics that consists of ∼102miniature mobile robots. The swarm experiments of collective following and collective evacuation demonstrate that AMS interactions not only empower the swarm to promptly respond to the transient perturbation but also strengthen the self-organization of collective motions in terms of temporal cognition.

show abstract

Section: Discussionmentioning

confidence: 99%

Perception of Motion Salience Shapes the Emergence of Collective Motions

Xiao,

Lei,

Zheng

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

Through Hawks’ Eyes: Synthetically Reconstructing the Visual Field of a Bird in Flight

et al. 2023

Self Cite

View full text Add to dashboard Cite

Birds of prey rely on vision to execute flight manoeuvres that are key to their survival, such as intercepting fast-moving targets or navigating through clutter. A better understanding of the role played by vision during these manoeuvres is not only relevant within the field of animal behaviour, but could also have applications for autonomous drones. In this paper, we present a novel method that uses computer vision tools to analyse the role of active vision in bird flight, and demonstrate its use to answer behavioural questions. Combining motion capture data from Harris’ hawks with a hybrid 3D model of the environment, we render RGB images, semantic maps, depth information and optic flow outputs that characterise the visual experience of the bird in flight. In contrast with previous approaches, our method allows us to consider different camera models and alternative gaze strategies for the purposes of hypothesis testing, allows us to consider visual input over the complete visual field of the bird, and is not limited by the technical specifications and performance of a head-mounted camera light enough to attach to a bird’s head in flight. We present pilot data from three sample flights: a pursuit flight, in which a hawk intercepts a moving target, and two obstacle avoidance flights. With this approach, we provide a reproducible method that facilitates the collection of large volumes of data across many individuals, opening up new avenues for data-driven models of animal behaviour.

show abstract