Evaluation of Hyperparameter-Optimization Approaches in an Industrial Federated Learning System

Abstract

Federated Learning (FL) decouples model training from the need for direct access to the data and allows organizations to collaborate with industry partners to reach a satisfying level of performance without sharing vulnerable business information. The performance of a machine learning algorithm is highly sensitive to the choice of its hyperparameters. In an FL setting, hyperparameter optimization poses new challenges. In this work, we investigated the impact of different hyperparameter optimization approaches in an FL system. In an effort to reduce communication costs, a critical bottleneck in FL, we investigated a local hyperparameter optimization approach that -in contrast to a global hyperparameter optimization approach -allows every client to have its own hyperparameter configuration. We implemented these approaches based on grid search and Bayesian optimization and evaluated the algorithms on the MNIST data set using an i.i.d. partition and on an Internet of Things (IoT) sensor based industrial data set using a non-i.i.d. partition.

Differentially Private Federated Bayesian Optimization with Distributed Exploration

“…In addition to our setting of FBO which can be equivalently called federated GP bandit, other sequential decision-making problems have also been extended to the federated setting, including federated multi-armed bandit [57,72], federated linear bandit [17], and federated reinforcement learning [22]. Lastly, federated hyperparameter tuning (i.e., hyperparameter tuning of ML models in the federated setting) has been attracting growing attention recently [26,35].…”