Privacy-Preserving Learning via Data and Knowledge Distillation

Faisal, Fahim; Leung, Carson K.; Mohammed, Noman; Wang, Yang

doi:10.1109/dsaa60987.2023.10302547

Search citation statements

Order By: Relevance

Paper Sections

Select...

Citation Types

Supporting

Mentioning

Contrasting

Year Published

2024

Publication Types

Select...

Other2

Article1

Relationship

Self Cite0

Independent3

Authors

Journals

Cited by 3 publications

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

A Metaverse Platform for Air Pollution Analysis in Supporting Smart and Sustainable City Development

Armijos,

Avagyan,

Leung

et al. 2024

2024 IEEE 11th International Conference on Data Science and Advanced Analytics (DSAA)

View full text Add to dashboard Cite

A Metaverse Platform for Air Pollution Analysis in Supporting Smart and Sustainable City Development

Armijos,

Avagyan,

Leung

et al. 2024

2024 IEEE 11th International Conference on Data Science and Advanced Analytics (DSAA)

View full text Add to dashboard Cite

A digital twin system to classify tornado climatology data for predicting future disasters

Bian,

Grenier,

Jiang

et al. 2024

2024 IEEE 11th International Conference on Data Science and Advanced Analytics (DSAA)

View full text Add to dashboard Cite

Bidirectional Decoupled Distillation for Heterogeneous Federated Learning

Song,

Yan,

et al. 2024

Entropy

View full text Add to dashboard Cite

Federated learning enables multiple devices to collaboratively train a high-performance model on the central server while keeping their data on the devices themselves. However, due to the significant variability in data distribution across devices, the aggregated global model’s optimization direction may differ from that of the local models, making the clients lose their personality. To address this challenge, we propose a Bidirectional Decoupled Distillation For Heterogeneous Federated Learning (BDD-HFL) approach, which incorporates an additional private model within each local client. This design enables mutual knowledge exchange between the private and local models in a bidirectional manner. Specifically, previous one-way federated distillation methods mainly focused on learning features from the target class, which limits their ability to distill features from non-target classes and hinders the convergence of local models. To solve this limitation, we decompose the network output into target and non-target class logits and distill them separately using a joint optimization of cross-entropy and decoupled relative-entropy loss. We evaluate the effectiveness of BDD-HFL through extensive experiments on three benchmarks under IID, Non-IID, and unbalanced data distribution scenarios. Our results show that BDD-HFL outperforms state-of-the-art federated distillation methods across five baselines, achieving at most 3% improvement in average classification accuracy on the CIFAR-10, CIFAR-100, and MNIST datasets. The experiments demonstrate the superiority and generalization capability of BDD-HFL in addressing personalization challenges in federated learning.

show abstract

Privacy-Preserving Learning via Data and Knowledge Distillation

Cited by 3 publications

References 15 publications

A Metaverse Platform for Air Pollution Analysis in Supporting Smart and Sustainable City Development

A Metaverse Platform for Air Pollution Analysis in Supporting Smart and Sustainable City Development

A digital twin system to classify tornado climatology data for predicting future disasters

Bidirectional Decoupled Distillation for Heterogeneous Federated Learning

Contact Info

Product

Resources

About