Architectural Patterns for the Design of Federated Learning Systems

Abstract

Federated learning has received fast-growing interests from academia and industry to tackle the challenges of data hungriness and privacy in machine learning. A federated learning system can be viewed as a large-scale distributed system with different components and stakeholders as numerous client devices participate in federated learning. Designing a federated learning system requires software system design thinking apart from machine learning knowledge. Although much effort has been put into federated learning from the machine learning technique aspects, the software architecture design concerns in building federated learning systems have been largely ignored. Therefore, in this paper, we present a collection of architectural patterns to deal with the design challenges of federated learning systems. Architectural patterns present reusable solutions to a commonly occurring problem within a given context during software architecture design. The presented patterns are based on the results of a systematic literature review and include three client management patterns, four model management patterns, three model training patterns, and four model aggregation patterns. The patterns are associated to particular state transitions in a federated learning model lifecycle, serving as a guidance for effective use of the patterns in the design of federated learning systems.

FLRA: A Reference Architecture for Federated Learning Systems

“…The deployment selector component examines the client devices and selects clients to receive the global model based on their data characteristics or applications. The deployment selector pattern has been applied in Azure Machine Learning 25 , Amazon SageMaker 26 , and Google Cloud 27 to improve model performance. The incentive registry com-ponent maintains all the client devices' incentives based on their contributions and agreed rates to motivate clients to contribute to the training.…”

“…A scalable production system for federated learning in the domain of mobile devices, based on TensorFlow described from a high-level perspective. A collection of architectural patterns for the design of federated learning systems are summarised and presented by [25]. There are also many architectures and adoptions of federated learning systems proposed by researchers for diverse applications.…”

“…Fig. 1: Federated Learning Overview [25]. This is a particular challenge in machine learning systems as the data that is ready for model training is often insufficient and they frequently suffer from "data hungriness issues".…”

“…A federated learning system can be viewed as a large-scale distributed system, involving different components and stakeholders with diverse requirements and constraints. Hence, developing a federated learning system requires both software system design thinking and machine learning knowledge [25]. Further, despite having various reference architectures for machine learning, big data, industrial IoT, and edge computing systems, to the best of our knowledge, there is still no reference architecture for an end-to-end federated learning system.…”

A Decision Model for Federated Learning Architecture Pattern Selection

“…In our previous work, we conducted a systematic literature review (SLR) on federated learning systems from the software engineering perspectives [12] to articulate the different software architectural challenges and propose approaches to tackle the challenges. We have summarised a set of software architectural patterns [13] to address the different requirements from different research articles and industrial practices. Despite having a collection of patterns, designers and architects may find it difficult to choose when and how to use a pattern.…”