Modeling deterministic echo state network with loop reservoir

Sun, Xiao Chuan; Cui, Hong Yan; Liu, Ren Ping; Chen, Jian Ya; Liu, Yun Jie

doi:10.1631/jzus.c1200069

Cited by 19 publications

(9 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Besides the basic connections, Sun et al [39,40] proposed the adjacent-feedback loop reservoir (ALR) based on SCR. ALR had a deterministic reservoir structure, in which the reservoir units were connected orderly in the loop manner and were also connected to the preceding one by adjacent feedback.…”

Section: Designs Of Reservoirmentioning

confidence: 99%

A Review of Designs and Applications of Echo State Networks

Sun¹,

Song²,

Hong³

et al. 2020

Preprint

View full text Add to dashboard Cite

Recurrent Neural Networks (RNNs) have demonstrated their outstanding ability in sequence tasks and have achieved state-of-the-art in wide range of applications, such as industrial, medical, economic and linguistic. Echo State Network (ESN) is simple type of RNNs and has emerged in the last decade as an alternative to gradient descent training based RNNs. ESN, with a strong theoretical ground, is practical, conceptually simple, easy to implement. It avoids non-converging and computationally expensive in the gradient descent methods. Since ESN was put forward in 2002, abundant existing works have promoted the progress of ESN, and the recently introduced Deep ESN model opened the way to uniting the merits of deep learning and ESNs. Besides, the combinations of ESNs with other machine learning models have also overperformed baselines in some applications. However, the apparent simplicity of ESNs can sometimes be deceptive and successfully applying ESNs needs some experience. Thus, in this paper, we categorize the ESN-based methods to basic ESNs, DeepESNs and combinations, then analyze them from the perspective of theoretical studies, network designs and specific applications. Finally, we discuss the challenges and opportunities of ESNs by summarizing the open questions and proposing possible future works.

show abstract

Section: Designs Of Reservoirmentioning

confidence: 99%

A Review of Designs and Applications of Echo State Networks

Sun¹,

Song²,

Hong³

et al. 2020

Preprint

View full text Add to dashboard Cite

show abstract

“…In terms of this, these three improved ESNs can reduce complexity and instability while achieving similar prediction accuracy to that of the classic ESN. Built upon SCR, Sun et al 12 proposed adjacent‐feedback loop reservoir (ALR), which incorporates the feedback connection between adjacent neurons in the simple cycle reservoir, to achieve better prediction performance over SCR.…”

Section: Related Workmentioning

confidence: 99%

“…Moreover, ESN has the faster training speed, because only the output weight needs to be trained by a simple linear regression algorithm. Therefore, including our previous work, ESN has been applied to network traffic prediction 11‐13 . However, the reservoir of ESN is randomly or specifically generated.…”

Section: Introductionmentioning

confidence: 99%

Network traffic prediction method based on echo state network with adaptive reservoir

et al. 2020

View full text Add to dashboard Cite

Network traffic prediction is of great significance to resource management in cyber‐physical systems (CPSs). In particular, network traffic is a nonlinear time series. Echo state network (ESN) is a new neural network with strong nonlinear processing capacity and short‐term memory capacity, and thus can achieve good performance in predicting nonlinear time series. However, network traffic has various characteristics such as self‐similarity, chaos, mutability. As the core of ESN, the reservoir will be fixed rather than adjustable once it is generated, which limits the prediction performance of ESN in different network traffic. To achieve universal excellent prediction performance, this paper proposes a new network traffic prediction method based on ESN with adaptive reservoir (ESN‐AR). First, the framework of ESN‐AR is constructed for network traffic prediction, in which the idea of generative adversarial network (GAN) is incorporated into ESN to adaptively adjust the reservoir. Specifically, ESN is used as the generative model to predict network traffic and feedforward neural network (FNN) is used as the discriminative model to distinguish between the real network traffic and the predicted network traffic. Second, the adversarial training algorithm of ESN‐AR is proposed to obtain the appropriate reservoir depending on the network traffic characteristics. Finally, ESN‐AR is applied to the prediction of three actual network traffic with different characteristics. Simulation results show that compared with the state‐of‐the‐art models, the proposed method achieves more accurate and stable prediction performance.

show abstract

“…out a y t -W x t x t = (13) where the prediction error ε=y(t)-W out x(t) is covariation orthogonality to the reservoir state x(t). Based on the decomposability of covariation [18], we can further convert the Eq.…”

Section: Following Quantitative Analysis Can Give Deeper Insights Intmentioning

confidence: 99%