A Hybrid Spiking Neural Network Reinforcement Learning Agent for Energy-Efficient Object Manipulation

Abstract

Due to the wide spread of robotics technologies in everyday activities, from industrial automation to domestic assisted living applications, cutting-edge techniques such as deep reinforcement learning are intensively investigated with the aim to advance the technological robotics front. The mandatory limitation of power consumption remains an open challenge in contemporary robotics, especially in real-case applications. Spiking neural networks (SNN) constitute an ideal compromise as a strong computational tool with low-power capacities. This paper introduces a spiking neural network actor for a baseline robotic manipulation task using a dual-finger gripper. To achieve that, we used a hybrid deep deterministic policy gradient (DDPG) algorithm designed with a spiking actor and a deep critic network to train the robotic agent. Thus, the agent learns to obtain the optimal policies for the three main tasks of the robotic manipulation approach: target-object reach, grasp, and transfer. The proposed method has one of the main advantages that an SNN possesses, namely, its neuromorphic hardware implementation capacity that results in energy-efficient implementations. The latter accomplishment is highly demonstrated in the evaluation results of the SNN actor since the deep critic network was exploited only during training. Aiming to further display the capabilities of the introduced approach, we compare our model with the well-established DDPG algorithm.

Computing of neuromorphic materials: an emerging approach for bioengineering solutions

“…Scaling up neuromorphic hardware systems to solve bigger, more complicated issues is one of the main objectives of. [276][277][278] The number of neurons and synapses that can currently be supported by the majority of neuromorphic hardware implementations is constrained. Scalable architectures and interconnectivity strategies are being developed by researchers to support larger networks and support more powerful calculations.…”

Point-to-Spike Residual Learning for Energy-Efficient 3D Point Cloud Classification

“…To date, SNNs have been successfully applied to a variety of areas in the field of neuromorphic computing, such as visual image classification and tracking, robot decision control (Oikonomou, Kansizoglou, and Gasteratos 2023), etc. In these applications, SNNs achieve performance comparable to conventional ANNs with low power consumption and high biological rationality.…”

Exploring Spiking Neural Networks for Deep Reinforcement Learning in Robotic Tasks: A Comparative Study

“…As the field of SNNs in DRL is relatively recent, most of the work done uses off-policy methods not exploiting the full potential of SNNs [7][8][9][10][11][12][13][14][15][16][17][18][19][20] .…”