Analysis on Characteristics of Multi-Step Learning Echo State Networks for Nonlinear Time Series Prediction

Akiyama, Takanori; Tanaka, Gouhei

doi:10.1109/ijcnn.2019.8851876

Cited by 12 publications

(19 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…7(a) (see Supplementary Figure 1 for the RMSE and MAE). This is because the multi-step learning ESN is suited for approximating dynamical systems with strong nonlinearity rather than those with long-term memory as suggested in our previous study [16]. The computational efficiency of the multi-step learning ESN is confirmed in Fig.…”

Section: B the Narma Modelsupporting

confidence: 69%

“…The multi-step learning ESN is one of the multi-reservoir ESN models, which consists of multiple ESN modules and additional connections as illustrated in Fig. 2 [16]. The Nstep learning ESN contains N ESN modules, each including an input layer, a reservoir, and a readout.…”

Section: B Multi-step Learning Echo State Networkmentioning

confidence: 99%

“…In these existing models, the model output is given by a weighted sum of the states from all the reservoir modules, and the readout training is conducted in one step. As another approach, we have proposed the multi-step learning ESN for time series prediction [16], inspired by the hybrid model combining a knowledge-based model and an ESN [7]. The multi-step learning ESN consists of a series of multiple ESN modules with additional connections, which are trained step by step to reduce the output error.…”

Section: Introductionmentioning

confidence: 99%

“…In our previous study, we have conducted the performance analysis only for the 2-step case with two ESN modules [16]. We also derived a method to compute Lyapunov exponents of the 2-step learning ESN and evaluated the model performance considering memory-nonlinearity tradeoff [17].…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Computational Efficiency of Multi-Step Learning Echo State Networks for Nonlinear Time Series Prediction

Akiyama

Tanaka

2022

IEEE Access

Self Cite

View full text Add to dashboard Cite

The echo state network (ESN) is a representative model for reservoir computing, which has been mainly used for temporal pattern recognition. Recent studies have shown that multi-reservoir ESN models constructed with multiple reservoirs can enhance the potential of the ESN-based approach. In the present study, we investigate computational performance and efficiency of the multi-step learning ESN which is one of the multi-reservoir ESN models and characterized by step-by-step learning processes. We show that the time complexity of the training algorithm of the multi-step learning ESN is equal to or smaller than that of the standard ESN. Our numerical experiments demonstrate that the multi-step learning ESN can achieve better or comparable performance with much less computational time than the standard ESN in nonlinear time series prediction tasks. Moreover, we reveal how the model architecture of the multistep learning ESN is useful compared with other possible variant models. The step-by-step learning is applicable to general multi-reservoir systems and hardware for enhancement of their computational ability and efficiency.

show abstract

Section: B the Narma Modelsupporting

confidence: 69%

Section: B Multi-step Learning Echo State Networkmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Computational Efficiency of Multi-Step Learning Echo State Networks for Nonlinear Time Series Prediction

Akiyama

Tanaka

2022

IEEE Access

Self Cite

View full text Add to dashboard Cite

show abstract

“…In contrast, recent connectomics analysis has revealed several important structural features in brain networks, including the modular organization [4]. Studies on modular-reservoir networks have reported that performance and robustness can be increased compared to randomly-connected reservoirs [5][6][7]. For example, Klampfl et al showed that modular structures that code specific inputs self-organize in a reservoir network when STDP is implemented in synapses within the reservoir and that the computational capability of linear readout neurons was enhanced after the STDP-based tuning of the synaptic weights [6].…”

Section: Introductionmentioning

confidence: 98%

Modular networks of spiking neurons for applications in time-series information processing

Moriya

Yamamoto

Hirano‐Iwata

et al. 2020

NOLTA

View full text Add to dashboard Cite

Spiking neural networks with complex spatiotemporal dynamics support efficient information processing of time-series signals. Here, we investigate the relationship between complexity of network dynamics and modular topology of networks using numerical simulations and discuss their effect on the classification performance of spoken-digit recognition tasks. The results show that modular networks generate spatially complex dynamics in which partially and globally synchronous bursts coexist. The classification rate of the modular reservoir network was approximately 75%, a value of which was comparable to that of a random network. This was caused by the randomly-connection structure between the input-reservoir and reservoirreadout layers, thus appropriate inference methods and asymmetry of connections should be introduced to take advantage of the complex dynamics in modular networks.

show abstract

Recent advances in physical reservoir computing: A review

et al. 2019

Self Cite

View full text Add to dashboard Cite

Reservoir computing is a computational framework suited for temporal/sequential data processing. It is derived from several recurrent neural network models, including echo state networks and liquid state machines. A reservoir computing system consists of a reservoir for mapping inputs into a high-dimensional space and a readout for pattern analysis from the high-dimensional states in the reservoir. The reservoir is fixed and only the readout is trained with a simple method such as linear regression and classification. Thus, the major advantage of reservoir computing compared to other recurrent neural networks is fast learning, resulting in low training cost. Another advantage is that the reservoir without adaptive updating is amenable to hardware implementation using a variety of physical systems, substrates, and devices. In fact, such physical reservoir computing has attracted increasing attention in diverse fields of research. The purpose of this review is to provide an overview of recent advances in physical reservoir computing by classifying them according to the type of the reservoir. We discuss the current issues and perspectives related to physical reservoir computing, in order to further expand its practical applications and develop next-generation machine learning systems.

show abstract

Analysis on Characteristics of Multi-Step Learning Echo State Networks for Nonlinear Time Series Prediction

Cited by 12 publications

References 22 publications

Computational Efficiency of Multi-Step Learning Echo State Networks for Nonlinear Time Series Prediction

Computational Efficiency of Multi-Step Learning Echo State Networks for Nonlinear Time Series Prediction

Modular networks of spiking neurons for applications in time-series information processing

Recent advances in physical reservoir computing: A review

Contact Info

Product

Resources

About