Experimental Study of Reinforcement Learning in Mobile Robots Through Spiking Architecture of Thalamo-Cortico-Thalamic Circuitry of Mammalian Brain

Abstract

Summary In this paper, the behavioral learning of robots through spiking neural networks is studied in which the architecture of the network is based on the thalamo-cortico-thalamic circuitry of the mammalian brain. According to a variety of neurons, the Izhikevich model of single neuron is used for the representation of neuronal behaviors. One thousand and ninety spiking neurons are considered in the network. The spiking model of the proposed architecture is derived and prepared for the learning problem of robots. The reinforcement learning algorithm is based on spike-timing-dependent plasticity and dopamine release as a reward. It results in strengthening the synaptic weights of the neurons that are involved in the robot’s proper performance. Sensory and motor neurons are placed in the thalamus and cortical module, respectively. The inputs of thalamo-cortico-thalamic circuitry are the signals related to distance of the target from robot, and the outputs are the velocities of actuators. The target attraction task is used as an example to validate the proposed method in which dopamine is released when the robot catches the target. Some simulation studies, as well as experimental implementation, are done on a mobile robot named Tabrizbot. Experimental studies illustrate that after successful learning, the meantime of catching target is decreased by about 36%. These prove that through the proposed method, thalamo-cortical structure could be trained successfully to learn to perform various robotic tasks.

Video Surveillance Vehicle Detection Method Incorporating Attention Mechanism and YOLOv5

“…9, which clearly reflects the comparison of the detection accuracy of the four VD algorithms network, to better reflect the improvement of the detection accuracy of the VD algorithms, also the more common VD algorithms and YOLOv5 combined with the CA mechanism of the network for comparison. It can be clearly found that the overall trend of the four VD algorithms is similar, and the detection accuracy in bus classification is much higher than the other classifications, at more than 90%, and the detection accuracy of the four algorithms for car classification is lower all approximating 83%, which may be due to the relatively fixed bus shape with obvious signs [19][20][21]. The YOLOv5 performance combined with CA mechanism for VD algorithm is significantly greater than the others, with detection accuracy exceeding other algorithms by about 0.8 percentage points.…”

A Novel Approach for Target Attraction and Obstacle Avoidance of a Mobile Robot in Unknown Environments Using a Customized Spiking Neural Network

“…Several studies have investigated the use of SNNs in designing robot control systems. For example, Azimirad and Sani [14] analyzed using SNNs inspired by the brain's circuitry for robot behavioral learning. They found that teaching the spiking architecture of these circuits can lead to successful target attraction.…”

Discrete soft actor-critic with auto-encoder on vascular robotic system

“…Reinforcement learning (RL) is achieved by trial-and-error without prior labeled datasets and has been widely used in many fields including games, robots, and autonomous vehicles [22][23][24]. Some research uses RL algorithms to learn instrument-manipulation skills based on vectorized lowdimensional representations of VIS scenarios [21,[25][26][27].…”