Decentralized learning works: An empirical comparison of gossip learning and federated learning

Hegedűs, István; Danner, Gábor; Jelasity, Márk

doi:10.1016/j.jpdc.2020.10.006

Cited by 88 publications

(41 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In addition, peer-to-peer protocols are also employed in FL to remove the need of a central node [ 105 , 106 ]. Gossip learning [ 107 , 108 ] is an alternative learning framework to FL [ 109 , 110 , 111 , 112 ]. Under gossip learning framework, no central node is required, nodes on the network exchange and aggregate models directly.…”

Section: Discussionmentioning

confidence: 99%

Federated Quantum Machine Learning

Chen

Yoo

2021

Entropy

View full text Add to dashboard Cite

Distributed training across several quantum computers could significantly improve the training time and if we could share the learned model, not the data, it could potentially improve the data privacy as the training would happen where the data is located. One of the potential schemes to achieve this property is the federated learning (FL), which consists of several clients or local nodes learning on their own data and a central node to aggregate the models collected from those local nodes. However, to the best of our knowledge, no work has been done in quantum machine learning (QML) in federation setting yet. In this work, we present the federated training on hybrid quantum-classical machine learning models although our framework could be generalized to pure quantum machine learning model. Specifically, we consider the quantum neural network (QNN) coupled with classical pre-trained convolutional model. Our distributed federated learning scheme demonstrated almost the same level of trained model accuracies and yet significantly faster distributed training. It demonstrates a promising future research direction for scaling and privacy aspects.

show abstract

Section: Discussionmentioning

confidence: 99%

Federated Quantum Machine Learning

Chen

Yoo

2021

Entropy

View full text Add to dashboard Cite

show abstract

“…Unfortunately, while gossip learning has been shown to be applicable to many different ML workloads [22][23][24], no study that we are aware of tested it in actual physical environments or evaluated its practical use for large-scale deep learning training. However, recent studies suggest that gossip learning compares favorably to federated learning [25] and that it can be extended to work in constrained and highly heterogeneous environments [26].…”

Section: Decentralized Machine Learningmentioning

confidence: 99%

PDS²: A user-centered decentralized marketplace for privacy preserving data processing

Giaretta

Savvidis

Marchioro

et al. 2021

2021 IEEE 37th International Conference on Data Engineering Workshops (ICDEW)

View full text Add to dashboard Cite

We envision PDS 2 , a decentralized data marketplace in which consumers submit their tasks to be run within the platform, on the data of willing providers. The goal of PDS 2 is to ensure that users maintain full control on their data and do not compromise their privacy, while being rewarded for the value that their data generates. In order to achieve this, our marketplace architecture employs blockchain technology, privacypreserving computation and decentralized machine learning.We then compare different potential solutions and identify the Ethereum blockchain, trusted execution environments and gossip learning as the most suitable for the implementation of PDS 2 . We also discuss the main open challenges that are left to tackle and possible directions for future work.

show abstract

“…With the emergence of infrastructure-less network scenarios, the optimization carried out in FL is moving out of the central server to be handled in a decentralized manner [15]. One of the first methods used to carry out server-less FL is based on the Gossip algorithm [16], which has been applied for deep learning (DL) distributed training with remarkable success [17]. Other solutions such as the one presented in [18] advocate for hybrid settings, where clients can use device-to-device (D2D) links to aggregate information from nodes away from the base station (BS).…”

Section: Related Workmentioning

confidence: 99%

Analysis and Evaluation of Synchronous and Asynchronous FLchain

Wilhelmi¹,

Giupponi²,

Dini³

2021

Preprint

View full text Add to dashboard Cite

Motivated by the heterogeneous nature of devices participating in large-scale Federated Learning (FL) optimization, we focus on an asynchronous server-less FL solution empowered by Blockchain (BC) technology. In contrast to mostly adopted FL approaches, which assume synchronous operation, we advocate an asynchronous method whereby model aggregation is done as clients submit their local updates. The asynchronous setting fits well with the federated optimization idea in practical large-scale settings with heterogeneous clients. Thus, it potentially leads to higher efficiency in terms of communication overhead and idle periods. To evaluate the learning completion delay of BC-enabled FL, we provide an analytical model based on batch service queue theory. Furthermore, we provide simulation results to assess the performance of both synchronous and asynchronous mechanisms. Important aspects involved in the BC-enabled FL optimization, such as the network size, link capacity, or user requirements, are put together and analyzed. As our results show, the synchronous setting leads to higher prediction accuracy than the asynchronous case. Nevertheless, asynchronous federated optimization provides much lower latency in many cases, thus becoming an appealing FL solution when dealing with large data sets, tough timing constraints (e.g., near-real-time applications), or highly varying training data.

show abstract

Decentralized learning works: An empirical comparison of gossip learning and federated learning

Cited by 88 publications

References 9 publications

Federated Quantum Machine Learning

Federated Quantum Machine Learning

PDS²: A user-centered decentralized marketplace for privacy preserving data processing

Analysis and Evaluation of Synchronous and Asynchronous FLchain

Contact Info

Product

Resources

About

Decentralized learning works: An empirical comparison of gossip learning and federated learning

Cited by 88 publications

References 9 publications

Federated Quantum Machine Learning

Federated Quantum Machine Learning

PDS2: A user-centered decentralized marketplace for privacy preserving data processing

Analysis and Evaluation of Synchronous and Asynchronous FLchain

Contact Info

Product

Resources

About

PDS²: A user-centered decentralized marketplace for privacy preserving data processing