Communication Efficient Federated Generalized Tensor Factorization for Collaborative Health Data Analytics

Ma, Jing; Zhang, Qiuchen; Lou, Jian; Xiong, Li; Ho, Joyce C.

doi:10.1145/3442381.3449832

Cited by 21 publications

(11 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Liu et al [21] adapt trained models with similar data distributions to achieve better personalization results for FL. As for applications, researchers utilize FL to help query suggestion [50], keyboard prediction [11], health data analytics [22], human activity recognition [16], and user modeling [43]. As for empirical studies, Yang et al [48] conduct the first empirical study to characterize the impacts of heterogeneity in FL.…”

Section: Related Workmentioning

confidence: 99%

“…Recently, the emerging popular Federated Learning (FL) [25] mitigates some of those concerns, where data are kept locally and local confidentiality issues are addressed [32]. Although FL has drawn numerous attention from researchers and developers [16,21,22,43,48,51], it uses central custodians to keep model parameters, which could still be attacked to infer users' identities and interests [3,27,33,35,38], even with privacy-preserving Deep Learning (DL) techniques. Besides, the star-shaped architecture of FL damages fault tolerance.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Demystifying Swarm Learning: A New Paradigm of Blockchain-based Decentralized Federated Learning

Han¹,

Ma²,

Han³

2022

Preprint

View full text Add to dashboard Cite

Federated learning (FL) is an emerging promising privacy-preserving machine learning paradigm and has raised more and more attention from researchers and developers. FL keeps users' private data on devices and exchanges the gradients of local models to cooperatively train a shared Deep Learning (DL) model on central custodians. However, the security and fault tolerance of FL have been increasingly discussed, because its central custodian mechanism or star-shaped architecture can be vulnerable to malicious attacks or software failures. To address these problems, Swarm Learning (SL) introduces a permissioned blockchain to securely onboard members and dynamically elect the leader, which allows performing DL in an extremely decentralized manner. Compared with tremendous attention to SL, there are few empirical studies on SL or blockchain-based decentralized FL, which provide comprehensive knowledge of best practices and precautions of deploying SL in real-world scenarios. Therefore, we conduct the first comprehensive study of SL to date, to fill the knowledge gap between SL deployment and developers, as far as we are concerned. In this paper, we conduct various experiments on 3 public datasets of 5 research questions, present interesting findings, quantitatively analyze the reasons behind these findings, and provide developers and researchers with practical suggestions. The findings have evidenced that SL is supposed to be suitable for most application scenarios, no matter whether the dataset is balanced, polluted, or biased over irrelevant features.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Demystifying Swarm Learning: A New Paradigm of Blockchain-based Decentralized Federated Learning

Han¹,

Ma²,

Han³

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Their algorithms also only used compression in the uplink direction. The work in [33] extended the analysis in [19] to cover the multi-block case. However, [33] still deals with unconstrained optimization; the compression was also only for Algorithm 1: CuteMaxVar // At the nodes…”

Section: Convergence Analysismentioning

confidence: 99%

“…First, observe that the update rule in (33) implies that there exists a Λ (r+1) and λ (r+1) such that the following optimality condition holds…”

Section: Supplementary Materials Of "Communication-efficient Distribu...mentioning

confidence: 99%

Communication-Efficient Distributed Linear and Deep Generalized Canonical Correlation Analysis

Shrestha

2021

Preprint

View full text Add to dashboard Cite

Classic and deep learning-based generalized canonical correlation analysis (GCCA) algorithms seek low-dimensional common representations of data entities from multiple "views" (e.g., audio and image) using linear transformations and neural networks, respectively. When the views are acquired and stored at different locations, organizations and edge devices, computing GCCA in a distributed, parallel and efficient manner is wellmotivated. However, existing distributed GCCA algorithms may incur prohitively high communication overhead. This work puts forth a communication-efficient distributed framework for both linear and deep GCCA under the maximum variance (MAX-VAR) paradigm. The overhead issue is addressed by aggressively compressing (via quantization) the exchanging information between the distributed computing agents and a central controller. Compared to the unquantized version, the proposed algorithm consistently reduces the communication overhead by about 90% with virtually no loss in accuracy and convergence speed. Rigorous convergence analyses are also presented-which is a nontrivial effort since no existing generic result from quantized distributed optimization covers the special problem structure of GCCA. Our result shows that the proposed algorithms for both linear and deep GCCA converge to critical points in a sublinear rate, even under heavy quantization and stochastic approximations. In addition, it is shown that in the linear MAX-VAR case, the quantized algorithm approaches a global optimum in a geometric rate-if the computing agents' updates meet a certain accuracy level. Synthetic and real data experiments are used to showcase the effectiveness of the proposed approach.

show abstract

“…FedAvg calculates the average weights of the models of all users and shares the weights with each user in the FL system [18]. For instance, Ma et al [19] devised a communicationefficient federated generalized tensor factorization for electronic health records. Liu et al [20] used a federated adaptation framework to leverage the sparsity property of neural networks for generating privacy-preserving representations.…”

Section: Introductionmentioning

confidence: 99%

An Efficient Federated Distillation Learning System for Multitask Time Series Classification

Xing

Xiao

et al. 2022

IEEE Trans. Instrum. Meas.

View full text Add to dashboard Cite

This paper proposes an efficient federated distillation learning system (EFDLS) for multi-task time series classification (TSC). EFDLS consists of a central server and multiple mobile users, where different users may run different TSC tasks. EFDLS has two novel components: a feature-based student-teacher (FBST) framework and a distance-based weights matching (DBWM) scheme. For each user, the FBST framework transfers knowledge from its teacher's hidden layers to its student's hidden layers via knowledge distillation, where the teacher and student have identical network structures. For each connected user, its student model's hidden layers' weights are uploaded to the EFDLS server periodically. The DBWM scheme is deployed on the server, with the least square distance used to measure the similarity between the weights of two given models. This scheme finds a partner for each connected user such that the user's and its partner's weights are the closest among all the weights uploaded. The server exchanges and sends back the user's and its partner's weights to these two users which then load the received weights to their teachers' hidden layers. Experimental results show that compared with a number of state-of-the-art federated learning algorithms, our proposed EFDLS wins 20 out of 44 standard UCR2018 datasets and achieves the highest mean accuracy (70.14%) on these datasets. In particular, compared with a single-task Baseline, EFDLS obtains 32/4/8 regarding 'win'/'tie'/'lose' and results in an improvement of approximately 4% in terms of mean accuracy.

show abstract

Communication Efficient Federated Generalized Tensor Factorization for Collaborative Health Data Analytics

Cited by 21 publications

References 32 publications

Demystifying Swarm Learning: A New Paradigm of Blockchain-based Decentralized Federated Learning

Demystifying Swarm Learning: A New Paradigm of Blockchain-based Decentralized Federated Learning

Communication-Efficient Distributed Linear and Deep Generalized Canonical Correlation Analysis

An Efficient Federated Distillation Learning System for Multitask Time Series Classification

Contact Info

Product

Resources

About