Finding the Gap: Neuromorphic Motion Vision in Cluttered Environments

Schoepe, Thorben; Janotte, Ella; Milde, Moritz B.; Bertrand, Olivier J. N.; Egelhaaf, Martin; Chicca, Elisabetta

doi:10.48550/arxiv.2102.08417

Cited by 4 publications

(4 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In insects, there are multiple neural layers of visual processing. Modeling these and other properties of the visual pathway is one of the obvious next steps for this work (83)(84)(85)(86). In short, the memory capacity of the network can be extended by tuning the learning parameters so that the weights grow slower, increasing the number of KCs, selectively learning only useful features, and/or introducing memory modulation and forgetting.…”

Section: Discussionmentioning

confidence: 99%

Neuromorphic sequence learning with an event camera on routes through vegetation

Zhu,

Mangan,

Webb

2023

Sci. Robot.

View full text Add to dashboard Cite

For many robotics applications, it is desirable to have relatively low-power and efficient onboard solutions. We took inspiration from insects, such as ants, that are capable of learning and following routes in complex natural environments using relatively constrained sensory and neural systems. Such capabilities are particularly relevant to applications such as agricultural robotics, where visual navigation through dense vegetation remains a challenging task. In this scenario, a route is likely to have high self-similarity and be subject to changing lighting conditions and motion over uneven terrain, and the effects of wind on leaves increase the variability of the input. We used a bioinspired event camera on a terrestrial robot to collect visual sequences along routes in natural outdoor environments and applied a neural algorithm for spatiotemporal memory that is closely based on a known neural circuit in the insect brain. We show that this method is plausible to support route recognition for visual navigation and more robust than SeqSLAM when evaluated on repeated runs on the same route or routes with small lateral offsets. By encoding memory in a spiking neural network running on a neuromorphic computer, our model can evaluate visual familiarity in real time from event camera footage.

show abstract

Section: Discussionmentioning

confidence: 99%

Neuromorphic sequence learning with an event camera on routes through vegetation

Zhu,

Mangan,

Webb

2023

Sci. Robot.

View full text Add to dashboard Cite

show abstract

“…Although the examples in this article focused primarily on vertebrates, the principles could be applied to modeling other organisms. Studies of the insect visual system have led to elegant, efficient solutions for robot navigation that could be deployed on neuromorphic hardware (Galluppi et al, 2014 ; Schoepe et al, 2021 ). The emphasis on vertebrates and especially the mammalian brain has been data-driven in part.…”

Section: Discussionmentioning

confidence: 99%

Design Principles for Neurorobotics

Krichmar

Hwu²

2022

Front. Neurorobot.

View full text Add to dashboard Cite

In their book “How the Body Shapes the Way We Think: A New View of Intelligence,” Pfeifer and Bongard put forth an embodied approach to cognition. Because of this position, many of their robot examples demonstrated “intelligent” behavior despite limited neural processing. It is our belief that neurorobots should attempt to follow many of these principles. In this article, we discuss a number of principles to consider when designing neurorobots and experiments using robots to test brain theories. These principles are strongly inspired by Pfeifer and Bongard, but build on their design principles by grounding them in neuroscience and by adding principles based on neuroscience research. Our design principles fall into three categories. First, organisms must react quickly and appropriately to events. Second, organisms must have the ability to learn and remember over their lifetimes. Third, organisms must weigh options that are crucial for survival. We believe that by following these design principles a robot's behavior will be more naturalistic and more successful.

show abstract

“…For example, the ANN version of EEM-SAN (iv) is 4.63 and 11.66 times more expensive than the SNN version (i) in terms of addition and multiplication operations, respectively. Prior works [14,17,48] have demonstrated that the computing complexity of the network is positively correlated with inference speed and energy consumption, especially when the network is implemented in neuromorphic device [3,35,43]. Therefore, our fully-SNN-based EEM-SAN can achieve much faster inference with much lower energy consumption than its ANN counterpart.…”

Section: Ablation Studymentioning

confidence: 93%

Event-Enhanced Multi-Modal Spiking Neural Network for Dynamic Obstacle Avoidance

Wang,

Dong,

Zhang

et al. 2023

Proceedings of the 31st ACM International Conference on Multimedia

View full text Add to dashboard Cite

Autonomous obstacle avoidance is of vital importance for an intelligent agent such as a mobile robot to navigate in its environment. Existing state-of-the-art methods train a spiking neural network (SNN) with deep reinforcement learning (DRL) to achieve energy-efficient and fast inference speed in complex/unknown scenes. These methods typically assume that the environment is static while the obstacles in real-world scenes are often dynamic. The movement of obstacles increases the complexity of the environment and poses a great challenge to the existing methods. In this work, we approach robust dynamic obstacle avoidance twofold. First, we introduce the neuromorphic vision sensor (i.e., event camera) to provide motion cues complementary to the traditional Laser depth data for handling dynamic obstacles. Second, we develop an DRL-based event-enhanced multimodal spiking actor network (EEM-SAN) that extracts information from motion events data via unsupervised representation learning and fuses Laser and event camera data with learnable thresholding. Experiments demonstrate that our EEM-SAN outperforms state-of-the-art obstacle avoidance methods by a significant margin, especially for dynamic obstacle avoidance.

show abstract

Finding the Gap: Neuromorphic Motion Vision in Cluttered Environments

Cited by 4 publications

References 0 publications

Neuromorphic sequence learning with an event camera on routes through vegetation

Neuromorphic sequence learning with an event camera on routes through vegetation

Design Principles for Neurorobotics

Event-Enhanced Multi-Modal Spiking Neural Network for Dynamic Obstacle Avoidance

Contact Info

Product

Resources

About