Robust and efficient target-tracking algorithms embedded on moving platforms, are a requirement for many computer vision and robotic applications. However, deployment of a real-time system is challenging, even with the computational power of modern hardware. As inspiration, we look to biological lightweight solutions-lightweight and low-powered flying insects. For example, dragonflies pursue prey and mates within cluttered, natural environments, deftly selecting their target amidst swarms. In our laboratory, we study the physiology and morphology of dragonfly 'small target motion detector' neurons likely to underlie this pursuit behaviour. Here we describe our insect-inspired tracking model derived from these data and compare its efficacy and efficiency with state-of-the-art engineering models. For model inputs, we use both publicly available video sequences, as well as our own task-specific dataset (small targets embedded within natural scenes). In the context of the tracking problem, we describe differences in object statistics within the video sequences. For the general dataset, our model often locks on to small components of larger objects, tracking these moving features. When input imagery includes small moving targets, for which our highly nonlinear filtering is matched, the robustness outperforms state-of-the-art trackers. In all scenarios, our insect-inspired tracker runs at least twice the speed of the comparison algorithms.

Although flying insects have limited visual acuity (approx. 18) and relatively small brains, many species pursue tiny targets against cluttered backgrounds with high success. Our previous computational model, inspired by electrophysiological recordings from insect 'small target motion detector' (STMD) neurons, did not account for several key properties described from the biological system. These include the recent observations of response 'facilitation' (a slow build-up of response to targets that move on long, continuous trajectories) and 'selective attention', a competitive mechanism that selects one target from alternatives. Here, we present an elaborated STMD-inspired model, implemented in a closed loop target-tracking system that uses an active saccadic gaze fixation strategy inspired by insect pursuit. We test this system against heavily cluttered natural scenes. Inclusion of facilitation not only substantially improves success for even short-duration pursuits, but it also enhances the ability to 'attend' to one target in the presence of distracters. Our model predicts optimal facilitation parameters that are static in space and dynamic in time, changing with respect to the amount of background clutter and the intended purpose of the pursuit. Our results provide insights into insect neurophysiology and show the potential of this algorithm for implementation in artificial visual systems and robotic applications.

Visual systems play a vital role in guiding the behaviour of animals. Understanding the visual information animals are able to acquire is therefore key to understanding their visually-mediated decision making. Compound eyes, the dominant eye type in arthropods, are inherently low-resolution structures. Their ability to resolve spatial detail depends on sampling resolution (interommatidial angle) and the quality of ommatidial optics. Current techniques for estimating interommatidial angles are difficult, and generally require in vivo measurements. Here, we present a new method for estimating interommatidial angles based on the detailed analysis of 3D Micro-CT images of fixed samples. Using custom-made MATLAB software we determine the optical axes of individual ommatidia and project these axes into the three-dimensional space around the animal. The combined viewing directions of all ommatidia, estimated from geometrical optics, allow us to estimate interommatidial angles and map the animal's sampling resolution across its entire visual field. The resulting topographic representations of visual acuity match very closely the previously published data obtained from both fiddler and grapsid crabs. However, the new method provides additional detail that was not previously detectable and reveals that fiddler crabs, rather than having a single horizontal visual streak as is common in flat world inhabitants, likely have two parallel streaks located just above and below the visual horizon. A key advantage of our approach is that it can be used on appropriately preserved specimens allowing the technique to be applied to animals such as deep-sea crustaceans that are inaccessible or unsuitable for in vivo approaches.

Our results provide insight into the neuronal mechanisms that underlie biological target detection and selection (from a moving platform), as well as highlight the effectiveness of our bio-inspired algorithm in an artificial visual system.

Vision in the midwater of the open ocean requires animals to perform visual tasks quite unlike those of any other environment. These tasks consist of detecting small, low contrast objects and point sources against a relatively dim and uniform background. Deep-sea animals have evolved many extraordinary visual adaptations to perform these tasks. Linking eye anatomy to specific selective pressures, however, is challenging, not least because of the many difficulties of studying deep-sea animals. Computational modelling of vision, based on detailed morphological reconstructions of animal eyes, along with underwater optics, offers a chance to understand the specific visual capabilities of individual visual systems. Prior to the work presented here, comprehensive models for apposition compound eyes in the mesopelagic, the dominant eye form of crustaceans, were lacking. We adapted a model developed for single-lens eyes and used it to examine how different parameters affect the model’s ability to detect point sources and extended objects. This new model also allowed us to examine spatial summation as a means to improve visual performance. Our results identify a trade-off between increased depth range over which eyes function effectively and increased distance at which extended objects can be detected. This trade-off is driven by the size of the ommatidial acceptance angle. We also show that if neighbouring ommatidia have overlapping receptive fields, spatial summation helps with all detection tasks, including the detection of bioluminescent point sources. By applying our model to the apposition compound eyes of Phronima, a mesopelagic hyperiid amphipod, we show that the specialisations of the large medial eyes of Phronima improve both the detection of point sources and of extended objects. The medial eyes outperformed the lateral eyes at every modelled detection task. We suggest that the small visual field size of Phronima’s medial eyes and the strong asymmetry between the medial and lateral eyes reflect Phronima’s need for effective vision across a large depth range and its habit of living inside a barrel. The barrel’s narrow aperture limits the usefulness of a large visual field and has allowed a strong asymmetry between the medial and lateral eyes. The model provides a useful tool for future investigations into the visual abilities of apposition compound eyes in the deep sea.