Clustering Algorithms in Echo State Networks

Ashour, Wesam M.; Abu-Issa, Abdallatif; Hellwich, Olaf

doi:10.14257/ijsip.2016.9.5.02

Cited by 2 publications

(2 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Yet another family of unsupervised learning algorithms are clustering techniques. In [24], the inverse weighted K -means (IWK) algorithm [25], [26] was proposed to initialize the weight matrices of an ESN. After randomly initializing the input weights, they applied IWK to the neuron inputs and adapted the input weights.…”

mentioning

confidence: 99%

Cluster-Based Input Weight Initialization for Echo State Networks

Steiner¹,

Jalalvand

Birkholz

2023

IEEE Trans. Neural Netw. Learning Syst.

View full text Add to dashboard Cite

Echo state networks (ESNs) are a special type of recurrent neural networks (RNNs), in which the input and recurrent connections are traditionally generated randomly, and only the output weights are trained. Despite the recent success of ESNs in various tasks of audio, image, and radar recognition, we postulate that a purely random initialization is not the ideal way of initializing ESNs. The aim of this work is to propose an unsupervised initialization of the input connections using the K -means algorithm on the training data. We show that for a large variety of datasets, this initialization performs equivalently or superior than a randomly initialized ESN while needing significantly less reservoir neurons. Furthermore, we discuss that this approach provides the opportunity to estimate a suitable size of the reservoir based on prior knowledge about the data.

show abstract

mentioning

confidence: 99%

Cluster-Based Input Weight Initialization for Echo State Networks

Steiner¹,

Jalalvand

Birkholz

2023

IEEE Trans. Neural Netw. Learning Syst.

View full text Add to dashboard Cite

show abstract

“…Yet another family of unsupervised learning algorithms are clustering techniques. In [18], the Inverse Weighted K-Means (IWK) algorithm [19], [20] was proposed to initialize the weight matrices of an ESN. After randomly initializing the input weights, they applied IWK to the neuron inputs and adapted the input weights.…”

Section: Introductionmentioning

confidence: 99%

Cluster-based Input Weight Initialization for Echo State Networks

Steiner,

Jalalvand,

Birkholz

2021

Preprint

View full text Add to dashboard Cite

Echo State Networks (ESNs) are a special type of recurrent neural networks (RNNs), in which the input and recurrent connections are traditionally generated randomly, and only the output weights are trained. Despite the recent success of ESNs in various tasks of audio, image and radar recognition, we postulate that a purely random initialization is not the ideal way of initializing ESNs. The aim of this work is to propose an unsupervised initialization of the input connections using the K-Means algorithm on the training data. We show that this initialization performs equivalently or superior than a randomly initialized ESN whilst needing significantly less reservoir neurons (2000 vs. 4000 for spoken digit recognition, and 300 vs. 8000 neurons for f0 extraction) and thus reducing the amount of training time. Furthermore, we discuss that this approach provides the opportunity to estimate the suitable size of the reservoir based on the prior knowledge about the data.

show abstract

Clustering Algorithms in Echo State Networks

Cited by 2 publications

References 22 publications

Cluster-Based Input Weight Initialization for Echo State Networks

Cluster-Based Input Weight Initialization for Echo State Networks

Cluster-based Input Weight Initialization for Echo State Networks

Contact Info

Product

Resources

About