Finding the Gap: Neuromorphic Motion Vision in Cluttered Environments

Abstract

Many animals meander in environments and avoid collisions. How the underlying neuronal machinery can yield robust behaviour in a variety of environments remains unclear. In the fly brain, motion-sensitive neurons indicate the presence of nearby objects and directional cues are integrated within an area known as the central complex. Such neuronal machinery, in contrast with the traditional stream-based approach to signal processing, uses an event-based approach, with events occurring when changes are sensed by the animal. Contrary to classical von Neumann computing architectures, event-based neuromorphic hardware is designed to process information asynchronously and in a distributed manner. Inspired by the fly brain, we model, for the first time, a neuromorphic closed-loop system mimicking essential behaviours observed in flying insects, such as meandering in clutter and crossing of gaps, both of which are also highly relevant for autonomous vehicles. We implemented our system both in software and on neuromorphic hardware. While moving through an environment, our agent perceives changes in its surroundings and uses this information for collision avoidance. The agent's manoeuvres result from a closed action-perception loop implementing probabilistic decision-making processes. This loop-closure is thought to have driven the development of neural circuitry in biological agents since the Cambrian explosion. In the fundamental quest to understand neural computation in artificial agents, we come closer to understanding and modelling biological intelligence by closing the loop also in neuromorphic systems. As a closed-loop system, our system deepens our understanding of processing in neural networks and their computations in both biological and artificial systems. With these investigations, we aim to set the foundations for neuromorphic intelligence in the future, moving towards leveraging the full potential of neuromorphic systems.

Embodied visual route following by an insect-inspired robot

“…In a cluttered or even structured environment, route recovery and generally route following can be relatively easy when the traversable area is well constrained, in which obstacle avoidance is more crucial. In an extremely structured environment, e.g., a single corridor, visual learning and memory can be considered redundant, as following a physical structure can be achieved by other lightweight and efficient (insect-inspired) mechanisms, e.g., using optic flow (Schoepe et al, 2024;Serres & Ruffier, 2017). Nevertheless, it is also possible and straightforward to extend differential-MB, so that, instead of being attracted by a familiar view in route following, the model forms a 'negative' memory that encourages itself to steer away from familiar but repulsive views.…”