On the application of reservoir computing networks for noisy image recognition

Abstract

Reservoir Computing Networks (RCNs) are a special type of single layer recurrent neural networks, in which the input and the recurrent connections are randomly generated and only the output weights are trained. Besides the ability to process temporal information, the key points of RCN are easy training and robustness against noise. Recently, we introduced a simple strategy to tune the parameters of RCNs. Evaluation in the domain of noise robust speech recognition proved that this method was effective. The aim of this work is to extend that study to the field of image processing, by showing that the proposed parameter tuning procedure is equally valid in the field of image processing and conforming that RCNs are apt at temporal modeling and are robust with respect to noise. In particular, we investigate the potential of RCNs in achieving competitive performance on the well-known MNIST dataset by following the aforementioned parameter optimizing strategy. Moreover, we achieve good noise robust recognition by utilizing such a network to denoise images and supplying them to a recognizer that is solely trained on clean images. The experiments demonstrate that the proposed RCN-based handwritten digit recognizer achieves an error rate of 0.81 percent on the clean test data of the MNIST benchmark and that the proposed RCN-based denoiser can effectively reduce the error rate on the various types of noise.

Multipitch tracking in music signals using Echo State Networks

“…ESNs offer a long-term memory that can keep information from the past for a long time. • They are quite robust against noise and unseen conditions [17].…”

“…Echo State Networks (ESNs), proposed by Herbert Jaeger [15], are a special kind of Recurrent Neural Networks (RNNs) and have achieved comparable results to CNNs in several recognition tasks, such as speech and image recognition [16], [17]. However, they are fairly new in the context of Music Information Retrieval (MIR).…”

ARDIS: a Swedish historical handwritten digit dataset

“…In [44], different models of CNNs have been discussed to achieve the highest accuracy rates for the handwritten digit recognition on NIST dataset. Many other deep learning methods have been designed and developed to obtain high recognition rate for different handwritten digit datasets [45][46][47][48].…”

Impact of coupling topology upon noise robustness of small optical reservoirs

“…Many nonlinear functions such as Mackey–Glass oscillator 11 , tanh 8 , sinusoidal 12 , sigmoid 13 , and chaotic function 14 can be used in the neurons. Several applications using RC have been demonstrated including speech/image recognition 15 , 16 , autonomous robots 17 , optical signal processing 18 , temporal information processing 19 , popularity prediction 20 , wind power ramp events predition 21 , attack detection of smart grids with wind power generators 22 , uncued brain-computer interface 23 , marking epileptic seizures on the intra-cranial electroencephalogram of rats 24 , non-linear time-series data analysis 25 , real-time audio processing 26 , real-time detection of epileptic seizures in animal models 27 , and noisy image recognition 28 , etc. Recently, we have also reported the deep learning of reservoir computing to predict the rainfall in Taiwan which is quite difficult to model theoretically in atmospheric science 29 .…”