Exploring deep echo state networks for image classification: a multi-reservoir approach

Abstract

Echo state networks (ESNs) belong to the class of recurrent neural networks and have demonstrated robust performance in time series prediction tasks. In this study, we investigate the capability of different ESN architectures to capture spatial relationships in images without transforming them into temporal sequences. We begin with three pre-existing ESN-based architectures and enhance their design by incorporating multiple output layers, customising them for a classification task. Our investigation involves an examination of the behaviour of these modified networks, coupled with a comprehensive performance comparison against the baseline vanilla ESN architecture. Our experiments on the MNIST data set reveal that a network with multiple independent reservoirs working in parallel outperforms other ESN-based architectures for this task, achieving a classification accuracy of 98.43%. This improvement on the classical ESN architecture is accompanied by reduced training times. While the accuracy of ESN-based architectures lags behind that of convolutional neural network-based architectures, the significantly lower training times of ESNs with multiple reservoirs operating in parallel make them a compelling choice for learning spatial relationships in scenarios prioritising energy efficiency and rapid training. This multi-reservoir ESN architecture overcomes standard ESN limitations regarding memory requirements and training times for large networks, providing more accurate predictions than other ESN-based models. These findings contribute to a deeper understanding of the potential of ESNs as a tool for image classification.

Energy-Efficient Edge and Cloud Image Classification with Multi-Reservoir Echo State Network and Data Processing Units

“…In the tests, we fed the network with 5 × 10 4 training examples, with the first 10 3 examples used for the initial transient phase. Once we calculated the weights of the output layer, we used the next 10 3 examples for testing, calculating the next value at each step.…”

“…This capacity to process vector inputs and generate specialized output layers enables the utilization of ESNs for image processing tasks. In our prior research [ 4 ], we investigated this modification in both the vanilla ESN architecture and the architectures proposed by Gallicchio [ 5 ]. Through this exploration, we examined the behaviors of these networks in image classification tasks using the widely recognized MNIST and FashionMNIST datasets.…”

“…The objectives of this study were several. Firstly, we aimed to extend the experiments conducted in our previous work [ 4 ], where we investigated the applicability of ESN-based architectures to image classification tasks beyond their conventional use in time series analysis. To achieve this, we focused on identifying the most efficient architecture for this task, which we have identified as the grouped ESN-based architecture [ 5 ].…”