Single SNN Architecture for Classical and Operant Conditioning Using Reinforcement Learning

Abstract

A bio-inspired robotic brain is presented where the same spiking neural network (SNN) can implement five variations of learning by conditioning (LC): classical conditioning (CC), and operant conditioning (OC) with positive/negative reinforcement/punishment. In all cases, the links between input stimuli, output actions, reinforcements and punishments are strengthened depending on the stability of the delays between them. To account for the parallel processing nature of neural networks, the SNN is implemented on a field-programmable gate array (FPGA), and the neural delays are extracted via an adaptation of the synapto-dendritic kernel adapting neuron (SKAN) model, for a low resource demanding FPGA implementation of the SNN. A custom robotic platform successfully tested the ability of the proposed architecture to implement the five LC behaviors. Hence, this work contributes to the engineering field by proposing a scalable low resource demanding architecture for adaptive systems, and the cognitive field by suggesting that both CC and OC can be modeled as a single cognitive architecture.

Neuromorphic Computing for Interactive Robotics: A Systematic Review

“…• HoLLiE arm [31] • FPGA ZEM4310, Emotive Epoc headset [23] • DVS, NAS [102] • UR3 with elbow and wrist [105] • Prosthetic hand [86] • Tobii Eye Tracker, YDLIDAR G4, Raspberry Pi 3 [100] Cognition and Learning [117] • Robotic hand, Myo armband, Epoc headset [25] [118] • Loihi, Neural Computer Stick2 [112] • Spartan-6, RGB sensor [113] • Raspberry Pi3, PiCamera, PiStrom [117] • Loihi, LIDAR [114] TABLE 6: Software/ simulators and neuron models for all the applications.…”

“…• SKAN [113] • SRM [111] VOLUME Furthermore Rast et al [28] made several enhancements to the basic networks and showed how they can be used to direct performance towards behaviourally relevant goals. They observed the behaviourally relevant STDP appears to contribute strongly to positive learning as compared to negative learning.…”

“…• Online [114][42] [115][120] • Offline [113] • STDP [25][118] [116][117] [111] • Reinforcement [116][117] [120][115] • Supervised/ Unsupervised [112] • Feedback-error learning [101] • Reinforcement [113] [42] • Synaptic learning [111] conditioning for a robot to learn the spatial map through reinforcement and punishment. The robot can learn the relationship between different visual stimuli irrespective of their pattern or location on the images.…”

“…Similarly, another article presents learning in robots where SNN can implement several variations of learning through classical conditioning with positive or negative reinforcement. The SNN model is implemented on Field-Programmable Gate Array (FPGA) and a Synapto-Dendritic Kernel Adapting Neuron (SKAN) model is used for neural delay [113].…”