Reinforcement learning with convolutional reservoir computing

Chang, Hanten; Futagami, Katsuya

doi:10.1007/s10489-020-01679-3

Cited by 20 publications

(8 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Notwithstanding these issues, we are optimistic that the ideas behind MARC can have far-reaching application beyond the focus of time series prediction presented in this work. Firstly, it may be the case that RCs trained for other tasks (such as control [45,46], signal classification [47,48], and anomaly detection [49,50]) or through reinforcement learning [51] can be similarly compressed with autoencoders so that new, related systems can be controlled or classified with greatly reduced training data requirements. Secondly, we have focused here on RCs-both because of our initial focus on time series prediction and due to attractive numerical properties-but the general scheme of Sec.…”

Section: Discussionmentioning

confidence: 99%

A Meta-learning Approach to Reservoir Computing: Time Series Prediction with Limited Data

Canaday,

Pomerance,

Girvan

2021

Preprint

View full text Add to dashboard Cite

Recent research has established the effectiveness of machine learning for data-driven prediction of the future evolution of unknown dynamical systems, including chaotic systems. However, these approaches require large amounts of measured time series data from the process to be predicted. When only limited data is available, forecasters are forced to impose significant model structure that may or may not accurately represent the process of interest. In this work, we present a Meta-learning Approach to Reservoir Computing (MARC), a data-driven approach to automatically extract an appropriate model structure from experimentally observed "related" processes that can be used to vastly reduce the amount of data required to successfully train a predictive model. We demonstrate our approach on a simple benchmark problem, where it beats the state of the art meta-learning techniques, as well as a challenging chaotic problem.

show abstract

Section: Discussionmentioning

confidence: 99%

A Meta-learning Approach to Reservoir Computing: Time Series Prediction with Limited Data

Canaday,

Pomerance,

Girvan

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Tong and Tanaka [49] took full advantage of an untrained CNN to transform raw image data into a set of small feature maps as a preprocessing step of RC and achieved a high classification accuracy with a much smaller number of trainable parameters compared with Schaetti's work. Then, a novel practical approach, called RL with a convolutional reservoir computing (RCRC) model [5] was proposed. A fixed random-weight CNN, used for extracting image feature maps, combined with an RC model, employed for extracting the time series features, is adopted for the evolution strategy of RL, which succeeded in decreasing computational cost to a large degree.…”

Section: Related Workmentioning

confidence: 99%

ViR:the Vision Reservoir

Wang¹,

Wang²,

Chen³

et al. 2021

Preprint

View full text Add to dashboard Cite

The most recent year has witnessed the success of applying the Vision Transformer (ViT) for image classification. However, there are still evidences indicating that ViT often suffers following two aspects, i) the high computation and the memory burden from applying the multiple Transformer layers for pre-training on a large-scale dataset, ii) the overfitting when training on small datasets from scratch. To address these problems, a novel method, namely, Vision Reservoir computing (ViR), is proposed here for image classification, as a parallel to ViT. By splitting each image into a sequence of tokens with a fixed length, the ViR constructs a pure reservoir with a nearly fully connected topology to replace the Transformer module in ViT. Two kinds of deep ViR models are subsequently proposed to enhance the network performance. Comparative experiments between the ViR and the ViT are carried out on several image classification benchmarks. Without any pre-training process, the ViR outperforms the ViT in terms of both model and computational complexity. Specifically, the number of parameters of the ViR is about 15% even 5% of the ViT, and the memory footprint is about 20% ∼ 40% of the ViT. The superiority of the ViR performance is explained by Small-World characteristics, Lyapunov exponents, and memory capacity.

show abstract

“…Reservoir computing has attracted significant attention in various research fields because it is capable of fast learning that results in reduced computational/training costs compared to other recurrent neural networks [19,20]. Reservoir computing is a computation framework used for information processing and supervised learning [21,22].…”

Section: Introductionmentioning

confidence: 99%

Photonic reinforcement learning based on optoelectronic reservoir computing

Kanno

Uchida

2022

Sci Rep

View full text Add to dashboard Cite

Reinforcement learning has been intensively investigated and developed in artificial intelligence in the absence of training data, such as autonomous driving vehicles, robot control, internet advertising, and elastic optical networks. However, the computational cost of reinforcement learning with deep neural networks is extremely high and reducing the learning cost is a challenging issue. We propose a photonic on-line implementation of reinforcement learning using optoelectronic delay-based reservoir computing, both experimentally and numerically. In the proposed scheme, we accelerate reinforcement learning at a rate of several megahertz because there is no required learning process for the internal connection weights in reservoir computing. We perform two benchmark tasks, CartPole-v0 and MountanCar-v0 tasks, to evaluate the proposed scheme. Our results represent the first hardware implementation of reinforcement learning based on photonic reservoir computing and pave the way for fast and efficient reinforcement learning as a novel photonic accelerator.

show abstract

Reinforcement learning with convolutional reservoir computing

Cited by 20 publications

References 26 publications

A Meta-learning Approach to Reservoir Computing: Time Series Prediction with Limited Data

A Meta-learning Approach to Reservoir Computing: Time Series Prediction with Limited Data

ViR:the Vision Reservoir

Photonic reinforcement learning based on optoelectronic reservoir computing

Contact Info

Product

Resources

About