A spiking neural network model for obstacle avoidance in simulated prosthetic vision

Abstract

Limited by visual percepts elicited by existing visual prothesis, it's necessary to enhance its functionality to fulfill some challenging tasks for the blind such as obstacle avoidance. This paper provides a new methodology for obstacle avoidance in simulated prosthetic vision by modelling and classifying spatiotemporal (ST) video data. The proposed methodology is based on a novel spiking neural network architecture, called NeuCube as a general framework for video data modelling in simulated prosthetic vision. As an integrated environment including spiking trains encoding, input variable mapping, unsupervised reservoir training and supervised classifier training, the NeuCube consists of a spiking neural network reservoir (SNNr) and a dynamic evolving spiking neural network classifier (deSNN). First input is captured by visual prothesis, then ST feature extraction is utilized in the low-resolution prosthetic vision generated by the prothesis. Finally such ST features are fed to the NeuCube to output classification result of obstacle analysis. Experiments on collected video data and comparison with other computational intelligence methods indicate promising results. The proposed NeuCube-based obstacle avoidance methodology provides useful guidance to the blind, thus improving the current prothesis and hopefully benefiting the future prothesis wearers.

Introductory Chapter: The “DNA Model” of Neurosciences and Computer Systems

“…Much progress has been made in the cardiovascular area for coronary abnormality detection inclusive of arrhythmias and infarctions [15,16], splicing circadian patterns with respect to sleep [17] and fatigue detection [18], prosthetic vision [19], and deep brain stimulators [20], among others. The human-robotic collaboration also forms an intricate and well-established research area for such application, given that commercialization of such products assisting production plants and surgeries are well documented.…”

Pattern Classification by Spiking Neural Networks Combining Self-Organized and Reward-Related Spike-Timing-Dependent Plasticity

“…Many spiking neural networks adopt a structure in which the spiking neural network with unsupervised learning is located before the output layer with supervised learning [9,20,21,22,33,13]. It has been previously shown that convolutional layers with unsupervised STDP synapses can detect the edges of applied images [13] and that selforganized STDP in the recurrent neural network, before output layers, enhance the learning efficiency for motor controls [33].…”

“…As another approach for learning for spiking neural networks, a learning approach using physiological spike-timing-dependent plasticity (STDP) has been used. For example, Kasabov et al proposed a spiking neural network for classification and time-series prediction, called NeuCube [9,20,21,22]. NeuCube is composed of an encoding module as the input layer, spiking neural clusters located in three-dimensional space as the inter-layer and a function module with supervised learning as the output layer.…”

Visual automatic obstacle avoidance technology research in unmanned vehicles

“…In order to ensure the follow-up obstacle avoidance accuracy of the unmanned vehicle, the target area outside border of vehicle travel need to be defined. In order to increase the efficiency and accuracy of obstacle positioning, it is necessary to specifically extract specific road lane information (Ge et al, 2017;Zi et al, 2015), while detect the edge of the road lane to achieve effective positioning of obstacles (Torres et al, 2015;Wang et al, 2015b;Zhang et al, 2016;Chai et al, 2016). Therefore, first-order derivation of Gaussian function is carried out by using functional derivative method, and the derivative is used as the best road lane line information.…”