Advances and Open Problems in Federated Learning

Kairouz, Peter; McMahan, H. Brendan; Avent, Brendan; Bellet, Aurélien; Bennis, Mehdi; Bhagoji, Arjun Nitin; Bonawitz, Kallista; Charles, Zachary; Cormode, Graham; Cummings, Rachel; D’Oliveira, Rafael G. L.; Eichner, Hubert; Rouayheb, Salim El; Evans, David; Gardner, Josh; Garrett, Zachary; Gascón, Adrià; Ghazi, Badih; Gibbons, Phillip B.; Gruteser, Marco; Harchaoui, Zaïd; He, Chuan; He, Lingxiao; Huo, Zhouyuan; Hutchinson, Ben; Hsu, Justin; Jäggi, Martin; Javidi, Tara; Joshi, Gauri; Khodak, Mikhail; Konečný, Jakub; Korolova, Aleksandra; Koushanfar, Farinaz; Koyejo, Sanmi; Lepoint, Tancrède; Liu, Yang; Mittal, Prateek; Mohri, Mehryar; Nock, Richard; Özgür, Ayfer; Pagh, Rasmus; Raykova, Mariana; Hu, Qi; Ramage, Daniel; Raskar, Ramesh; Song, Dawn; Song, Weikang; Stich, Sebastian U.; Sun, Ziteng; Suresh, Ananda Theertha; Tramèr, Florian; Vepakomma, Praneeth; Wang, Yunlong; Li, Xiong; Xu, Zheng; Yang, Qiang; Yu, Felix X.; Han, Yu

doi:10.48550/arxiv.1912.04977

Cited by 557 publications

(1,092 citation statements)

References 209 publications

Supporting

Mentioning

1,033

Contrasting

Unclassified

Order By: Relevance

“…One important class of approaches we did not make use of here is on-device learning and federated learning. In federated learning the model parameters are updated on device, and these updates communicated back to a central server (Kairouz et al, 2019), usually with a differential privacy guarantee to ensure that the updates strongly limit what can be inferred about an individual user (Geyer et al, 2017). On-device learning refers to approaches where the ad targeting model remains on the user device, and no user-specific information is communicated back to the server.…”

Section: Discussionmentioning

confidence: 99%

Challenges and approaches to privacy preserving post-click conversion prediction

O’Brien¹,

Thiagarajan²,

Das³

et al. 2022

Preprint

View full text Add to dashboard Cite

Online advertising has typically been more personalized than offline advertising, through the use of machine learning models and real-time auctions for ad targeting. One specific task, predicting the likelihood of conversion (i.e. the probability a user will purchase the advertised product), is crucial to the advertising ecosystem for both targeting and pricing ads. Currently, these models are often trained by observing individual user behavior, but, increasingly, regulatory and technical constraints are requiring privacy-preserving approaches. For example, major platforms are moving to restrict tracking individual user events across multiple applications, and governments around the world have shown steadily more interest in regulating the use of personal data. Instead of receiving data about individual user behavior, advertisers may receive privacy-preserving feedback, such as the number of installs of an advertised app that resulted from a group of users. In this paper we outline the recent privacy-related changes in the online advertising ecosystem from a machine learning perspective. We provide an overview of the challenges and constraints when learning conversion models in this setting. We introduce a novel approach for training these models that makes use of post-ranking signals. We show using offline experiments on real world data that it outperforms a model relying on opt-in data alone, and significantly reduces model degradation when no individual labels are available. Finally, we discuss future directions for research in this evolving area.

show abstract

Section: Discussionmentioning

confidence: 99%

Challenges and approaches to privacy preserving post-click conversion prediction

O’Brien¹,

Thiagarajan²,

Das³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…There are multiple challenges in federated learning [4]. Data skewness is the most popular among them.…”

Section: Introductionmentioning

confidence: 99%

Client Selection in Federated Learning under Imperfections in Environment

Rai

Kumari

Prasad

2022

View full text Add to dashboard Cite

Federated learning promises an elegant solution for learning global models across distributed and privacy-protected datasets. However, challenges related to skewed data distribution, limited computational and communication resources, data poisoning, and free riding clients affect the performance of federated learning. Selection of the best clients for each round of learning is critical in alleviating these problems. We propose a novel sampling method named the irrelevance sampling technique. Our method is founded on defining a novel irrelevance score that incorporates the client characteristics in a single floating value, which can elegantly classify the client into three numerical sign defined pools for easy sampling. It is a computationally inexpensive, intuitive and privacy preserving sampling technique that selects a subset of clients based on quality and quantity of data on edge devices. It achieves 50–80% faster convergence even in highly skewed data distribution in the presence of free riders based on lack of data and severe class imbalance under both Independent and Identically Distributed (IID) and Non-IID conditions. It shows good performance on practical application datasets.

show abstract

“…Despite the numerous FL-based applications in various domains (Kairouz et al, 2019) ranging from health to NLP, there are very little progress (Aggarwal et al, 2021;Bai et al, 2021) in training face recognition models with FL schemes. Unlike other tasks, parameters of the last classifier for a face recognition model are crucial for recognition performance but strongly associated with privacy.…”

Section: Introductionmentioning

confidence: 99%

Improving Federated Learning Face Recognition via Privacy-Agnostic Clusters

Meng¹,

Zhu²,

Ren³

et al. 2022

Preprint

View full text Add to dashboard Cite

The growing public concerns on data privacy in face recognition can be greatly addressed by the federated learning (FL) paradigm. However, conventional FL methods perform poorly due to the uniqueness of the task: broadcasting class centers among clients is crucial for recognition performances but leads to privacy leakage. To resolve the privacy-utility paradox, this work proposes PrivacyFace, a framework largely improves the federated learning face recognition via communicating auxiliary and privacy-agnostic information among clients. PrivacyFace mainly consists of two components: First, a practical Differentially Private Local Clustering (DPLC) mechanism is proposed to distill sanitized clusters from local class centers. Second, a consensus-aware recognition loss subsequently encourages global consensuses among clients, which ergo results in more discriminative features. The proposed framework is mathematically proved to be differentially private, introducing a lightweight overhead as well as yielding prominent performance boosts (e.g., +9.63% and +10.26% for TAR@FAR=1e-4 on IJB-B and IJB-C respectively). Extensive experiments and ablation studies on a large-scale dataset have demonstrated the efficacy and practicability of our method.

show abstract

Advances and Open Problems in Federated Learning

Cited by 557 publications

References 209 publications

Challenges and approaches to privacy preserving post-click conversion prediction

Challenges and approaches to privacy preserving post-click conversion prediction

Client Selection in Federated Learning under Imperfections in Environment

Improving Federated Learning Face Recognition via Privacy-Agnostic Clusters

Contact Info

Product

Resources

About