An experimental unification of reservoir computing methods

Verstraeten, David; Schrauwen, Benjamin; D’Haene, Michiel; Stroobandt, Dirk

doi:10.1016/j.neunet.2007.04.003

Cited by 943 publications

(705 citation statements)

References 20 publications

Supporting

Mentioning

694

Contrasting

Unclassified

Order By: Relevance

“…This point has been widely discussed and confirmed by studies on the network dynamics proving that a spectral radius close to 1 is an optimal value. However, we share the opinion of Verstraeten et al [49] who claim that ''for spiking neurons it has no influence at all''. In [43] Steil shows that a learning rule based on IP has the effect to expand the eigenvalues away from the center of the unit disk.…”

Section: Article In Presssupporting

confidence: 85%

Delay learning and polychronization for reservoir computing

Paugam-Moisy¹,

Martinez²,

Bengio³

2008

Neurocomputing

View full text Add to dashboard Cite

We propose a multi-timescale learning rule for spiking neuron networks, in the line of the recently emerging field of reservoir computing. The reservoir is a network model of spiking neurons, with random topology and driven by STDP (spike-time-dependent plasticity), a temporal Hebbian unsupervised learning mode, biologically observed. The model is further driven by a supervised learning algorithm, based on a margin criterion, that affects the synaptic delays linking the network to the readout neurons, with classification as a goal task. The network processing and the resulting performance can be explained by the concept of polychronization, proposed by Izhikevich [Polychronization: computation with spikes, Neural Comput. 18(2) (2006) 245-282], on physiological grounds. The model emphasizes that polychronization can be used as a tool for exploiting the computational power of synaptic delays and for monitoring the topology and activity of a spiking neuron network. r

show abstract

Section: Article In Presssupporting

confidence: 85%

Delay learning and polychronization for reservoir computing

Paugam-Moisy¹,

Martinez²,

Bengio³

2008

Neurocomputing

View full text Add to dashboard Cite

show abstract

“…Then, the learning side is concentrated on the readout part. This approach is based on the empirical observation that under certain hypothesis, a learning process restricted to the readout weights is often sufficient to obtain an excellent performance in many learning tasks [5]- [8].…”

Section: Description Of the Methods Proposedmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

“…The first two proposed RC models were Liquid State Machines (LSMs) [6] and Echo State Networks (ESNs) [7], both almost simultaneously published. The two types of models have been successfully applied in many problems achieving very good results in temporal and non-temporal learning tasks [5], [8], [9].In this paper, we study the problem of estimating PESQ scores in a context where the reference signal is not available, using the ESN model. The main idea is to capture the relation between certain network parameters that affect the perceived quality and their corresponding PESQ scores.…”

mentioning

confidence: 99%

See 1 more Smart Citation

Real-Time Estimation of Speech Quality through the Internet Using Echo State Networks

Basterrech¹,

Rubino²

2013

JACN

View full text Add to dashboard Cite

Abstract-Audio quality in the Internet can be strongly affected by network conditions. As a consequence, many techniques to evaluate it have been developed. In particular, the ITU-T adopted in 2001 a technique called Perceptual Evaluation of Speech Quality (PESQ) to automatically measuring speech quality. PESQ is a well-known and widely used procedure, providing in general an accurate evaluation of perceptual quality by comparing the original and received voice sequences.One obvious inherent limitation of PESQ is, thus, that it requires the original signal (we say the reference), to make its evaluation. This precludes the use of PESQ for assessing the perceived quality in real-time, as the reference is in general not available.In this paper, we describe a procedure for estimating PESQ output working only with measures taken on the network state and properties of the communication system, without any use of the reference. It is based on the use of statistical learning techniques. Specifically, we rely on recent ideas for learning with specific types of neural networks, known under the name of Echo State Networks (ESNs), a member of the class of Reservoir Computing systems. These tools have been proven to be very efficient and robust in many learning tasks. The experimental results obtained show the good accuracy of the resulting procedure, and its capability to give its estimations of speech quality in a real-time context. This allows putting our measuring modules in future Internet applications or services based on voice transmission, for instance for control purposes.Index Terms-Quality assessment, speech quality, echo state networks, reservoir computing. I. INTRODUCTIONMeasuring the quality of a voice signal transmitted over the Internet is an important topic today, and one of main available tools for this purpose is the Perceptual Evaluation of Speech Quality (PESQ) method accepted in 2001 as the ITU-T objective speech quality measurement standard P.862 [1]. The network conditions vary over time, and in many contexts, several different factors lead to losses, which in turn lead to degradations in the perception of the quality by the users. PESQ analyzes this quality by comparing the received signal with the original speech sequence. For this reason, we say that it is a "full reference" technique, the reference being the original signal. Researchers in many areas use PESQ and the tool has been widely diffused in commercial measurement products. Recently, the ITU started to update its voice Manuscript received March 1, 2013; revised April 15, 2013. This work was supported in part by the European Celtic Project "QuEEN".S. Basterrech is with the University of Rennes 2, Rennes, France (e-mail: Sebastian.Basterrechtiscordio@etudiant.uhb.fr).G. Rubino is with the National Institute for Research in Computer Science and Control (INRIA Rennes -Bretagne Atlantique), Rennes, France (e-mail: Gerardo.Rubino@inria.fr). assessment recommendations by promoting the new P.863 standard Perceptual Objective Listening Quality Asse...

show abstract

“…Each of the layers in the proposed hierarchy uses a random dynamical system of which only the readout layer is trained (eg. Reservoir Computing systems [7]). On the lowest level, the fastest timescale, the passive compliance of the leg interacts with the environment.…”

Section: Introductionmentioning

confidence: 99%

Towards a Neural Hierarchy of Time Scales for Motor Control

Waegeman

wyffels

Schrauwen

2012

From Animals to Animats 12

View full text Add to dashboard Cite

Abstract. Animals show remarkable rich motion skills which are still far from realizable with robots. Inspired by the neural circuits which generate rhythmic motion patterns in the spinal cord of all vertebrates, one main research direction points towards the use of central pattern generators in robots. On of the key advantages of this, is that the dimensionality of the control problem is reduced. In this work we investigate this further by introducing a multi-timescale control hierarchy with at its core a hierarchy of recurrent neural networks. By means of some robot experiments, we demonstrate that this hierarchy can embed any rhythmic motor signal by imitation learning. Furthermore, the proposed hierarchy allows the tracking of several high level motion properties (e.g.: amplitude and offset), which are usually observed at a slower rate than the generated motion. Although these experiments are preliminary, the results are promising and have the potential to open the door for rich motor skills and advanced control.

show abstract

An experimental unification of reservoir computing methods

Cited by 943 publications

References 20 publications

Delay learning and polychronization for reservoir computing

Delay learning and polychronization for reservoir computing

Real-Time Estimation of Speech Quality through the Internet Using Echo State Networks

Towards a Neural Hierarchy of Time Scales for Motor Control

Contact Info

Product

Resources

About