An End-to-End Spiking Neural Network Platform for Edge Robotics: From Event-Cameras to Central Pattern Generation

Lele, Ashwin Sanjay; Fang, Yan; Ting, Justin; Raychowdhury, Arijit

doi:10.1109/tcds.2021.3097675

Cited by 19 publications

(9 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Thanks to their desirable characteristics, SNNs have gathered interest in a range of robotics applications, including control [5]- [8], manipulation [9], [10], scene understanding [11], and object tracking [12]. Key works that use spiking networks for robot localization, the task considered in this paper, include an energy-efficient uni-dimensional SLAM [39], a robot navigation controller system [40], a pose estimation and mapping system [41], and models of the place, grid and border cells of rat hippocampus [42] based on RatSLAM [43].…”

Section: A Spiking Neural Network In Robotics Researchmentioning

confidence: 99%

“…SNNs have thus been used in a number of robotics applications [5]- [12], including the visual place recognition (VPR) task [13], [14] that is considered in this paper. A VPR system has to find the matching reference image given a query image of a place, with the difficulty that the appearance of the query image can differ significantly from the reference image due to change in season, time of the day, or weather conditions [15]- [18].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Ensembles of Compact, Region-specific & Regularized Spiking Neural Networks for Scalable Place Recognition

Hussaini¹,

Milford²,

Fischer³

2022

Preprint

View full text Add to dashboard Cite

Spiking neural networks have significant potential utility in robotics due to their high energy efficiency on specialized hardware, but proof-of-concept implementations have not yet typically achieved competitive performance or capability with conventional approaches. In this paper, we tackle one of the key practical challenges of scalability by introducing a novel modular ensemble network approach, where compact, localized spiking networks each learn and are solely responsible for recognizing places in a local region of the environment only. This modular approach creates a highly scalable system. However, it comes with a high-performance cost where a lack of global regularization at deployment time leads to hyperactive neurons that erroneously respond to places outside their learned region. Our second contribution introduces a regularization approach that detects and removes these problematic hyperactive neurons during the initial environmental learning phase. We evaluate this new scalable modular system on benchmark localization datasets Nordland and Oxford RobotCar, with comparisons to both standard techniques NetVLAD and SAD, and a previous spiking neural network system. Our system substantially outperforms the previous SNN system on its small dataset, but also maintains performance on 27 times larger benchmark datasets where the operation of the previous system is computationally infeasible, and performs competitively with the conventional localization systems.

show abstract

Section: A Spiking Neural Network In Robotics Researchmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Ensembles of Compact, Region-specific & Regularized Spiking Neural Networks for Scalable Place Recognition

Hussaini¹,

Milford²,

Fischer³

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…• HoLLiE arm [31] • FPGA ZEM4310, Emotive Epoc headset [23] • DVS, NAS [102] • UR3 with elbow and wrist [105] • Prosthetic hand [86] • Tobii Eye Tracker, YDLIDAR G4, Raspberry Pi 3 [100] Cognition and Learning [117] • Robotic hand, Myo armband, Epoc headset [25] [118] • Loihi, Neural Computer Stick2 [112] • Spartan-6, RGB sensor [113] • Raspberry Pi3, PiCamera, PiStrom [117] • Loihi, LIDAR [114] TABLE 6: Software/ simulators and neuron models for all the applications.…”

Section: Soundmanmentioning

confidence: 99%

“…The experimental results show that the robot was able to replicate the gait pattern generated through the user's mental activity with a slight delay. Similarly, Lele et al [102] used CPG and coupled it with Dynamic Vision Sensor (DVS) for a prey-tracking scenario in the close-loop robotic system. Here the legs of a Hexapod robot are controlled by a network of spiking neurons.…”

Section: Motor Control Applicationsmentioning

confidence: 99%

Neuromorphic Computing for Interactive Robotics: A Systematic Review

2022

View full text Add to dashboard Cite

Modelling functionalities of the brain in human-robot interaction contexts requires a real-time understanding of how each part of a robot (motors, sensors, emotions, etc.) works and how they interact all together to accomplish complex behavioural tasks while interacting with the environment. Human brains are very efficient as they process the information using event-based impulses also known as spikes, which make living creatures very efficient and able to outperform current mainstream robotic systems in almost every task that requires real-time interaction. In recent years, combined efforts by neuroscientists, biologists, computer scientists and engineers make it possible to design biologically realistic hardware and models that can endow the robots with the required human-like processing capability based on neuromorphic computing and Spiking Neural Network (SNN). However, while some attempts have been made, a comprehensive combination of neuromorphic computing and robotics is still missing. In this article, we present a systematic review of neuromorphic computing applications for socially interactive robotics. We first introduce the basic principles, models and architectures of neuromorphic computation. The remaining articles are classified according to the applications they focus on. Finally, we identify the potential research topics for fully integrated socially interactive neuromorphic robots.

show abstract

“…Another neuro-inspired system exploration is the first autonomous sensing-to-actuation end-to-end spike-only processing pipeline for hexapod robots [35]. The goal is to demonstrate the functionality of spike-only processing and evaluate the potential of event-driven processing modalities.…”

Section: Neuro-inspired End-to-end Spike-only Processingmentioning

confidence: 99%

Circuit and System Technologies for Energy-Efficient Edge Robotics: (Invited Paper)

Wan

Lele

Raychowdhury

2022

2022 27th Asia and South Pacific Design Automation Conference (ASP-DAC)

Self Cite

View full text Add to dashboard Cite

As we march towards the age of ubiquitous intelligence, we note that AI and intelligence are progressively moving from the cloud to the edge. The success of Edge-AI is pivoted on innovative circuits and hardware that can enable inference and limited learning in resource-constrained edge autonomous systems. This paper introduces a series of ultra-low-power accelerator and system designs on enabling the intelligence in edge robotic platforms, including reinforcement learning neuromorphic control, swarm intelligence, and simultaneous mapping and localization. We put an emphasis on the impact of the mixedsignal circuit, neuro-inspired computing system, benchmarking and software infrastructure, as well as algorithm-hardware codesign to realize the most energy-efficient Edge-AI ASICs for the next-generation intelligent and autonomous systems.

show abstract

An End-to-End Spiking Neural Network Platform for Edge Robotics: From Event-Cameras to Central Pattern Generation

Cited by 19 publications

References 38 publications

Ensembles of Compact, Region-specific & Regularized Spiking Neural Networks for Scalable Place Recognition

Ensembles of Compact, Region-specific & Regularized Spiking Neural Networks for Scalable Place Recognition

Neuromorphic Computing for Interactive Robotics: A Systematic Review

Circuit and System Technologies for Energy-Efficient Edge Robotics: (Invited Paper)

Contact Info

Product

Resources

About