Federated Adaptation of Reservoirs via Intrinsic Plasticity

Abstract

We propose a novel algorithm for performing federated learning with Echo State Networks (ESNs) in a client-server scenario. In particular, our proposal focuses on the adaptation of reservoirs by combining Intrinsic Plasticity with Federated Averaging. The former is a gradientbased method for adapting the reservoir's non-linearity in a local and unsupervised manner, while the latter provides the framework for learning in the federated scenario. We evaluate our approach on real-world datasets from human monitoring, in comparison with the previous approach for federated ESNs existing in literature. Results show that adapting the reservoir with our algorithm provides a significant improvement on the performance of the global model.

Communication-Efficient Ridge Regression in Federated Echo State Networks

“…In both datasets, we chunked the sequence in sections of 700 and 200 timesteps for WESAD and HHAR respectively. Similarly to [2], we employed a clientwise training-validation-test split of the dataset, which is 9-3-3 and 5-2-2 for WESAD and HHAR respectively. We conducted our experiments by involving an incremental number of training clients, i.e., 25%, 50%, 75% and 100%.…”

“…This has led to the emergence of federated learning, a decentralized approach where models are trained on data distributed across multiple devices without the need to transfer the data to a central server. In such setting, Federated Echo State Networks (ESNs) were proven to be effective thanks to their capability to efficiently handle temporal data, as well as for the low computational cost of the learning algorithms [1,2]. In this work, we aim to further improve FedRR [1], an algorithm that performs an exact computation of the global readout in a federated setting, towards communication efficiency.…”