Cluster-driven Graph Federated Learning over Multiple Domains

Abstract

Federated Learning (FL) deals with learning a central model (i.e. the server) in privacy-constrained scenarios, where data are stored on multiple devices (i.e. the clients). The central model has no direct access to the data, but only to the updates of the parameters computed locally by each client. This raises a problem, known as statistical heterogeneity, because the clients may have different data distributions (i.e. domains). This is only partly alleviated by clustering the clients. Clustering may reduce heterogeneity by identifying the domains, but it deprives each cluster model of the data and supervision of others.Here we propose a novel Cluster-driven Graph Federated Learning (FedCG). In FedCG, clustering serves to address statistical heterogeneity, while Graph Convolutional Networks (GCNs) enable sharing knowledge across them. FedCG: i. identifies the domains via an FL-compliant clustering and instantiates domain-specific modules (residual branches) for each domain; ii. connects the domain-specific modules through a GCN at training to learn the interactions among domains and share knowledge; and iii. learns to cluster unsupervised via teacher-student classifier-training iterations and to address novel unseen test domains via their domain soft-assignment scores. Thanks to the unique interplay of GCN over clusters, FedCG achieves the state-of-theart on multiple FL benchmarks.

Optimality and Stability in Federated Learning: A Game-theoretic Approach

“…For example, hierarchical federated learning adds an additional layer of hierarchical structure to federated learning, which could be used to reduce latency or to cluster together similar players (Lin et al [2018], Liu et al [2020). Many other works also relate to clustering, such as (Lee et al [2020], Sattler et al [2020], Shlezinger et al [2020], Wang et al [2020], Duan et al, Jamali-Rad et al [2021], Caldarola et al [2021]). These works, which tend to be more applied than our work, may also differ in that they analyze situations where additional information is known, such as the data distribution at each location.…”