Hiding in the Crowd: Federated Data Augmentation for On-Device Learning

Jeong, Eunjeong; Oh, Seungeun; Park, Jihong; Kim, Hyesung; Bennis, Mehdi; Kim, Seong-Lyun

doi:10.1109/mis.2020.3028613

Cited by 19 publications

(18 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Second, FL fundamentally requires ensuring privacy within each agent, which commonly implies but is not limited to maintaining data samples private. For instance, the information on possessed class labels for classification tasks should also be kept private [21]. However, because of their omniscient viewpoint, most existing personalized and decentralized learning schemes do not thoroughly preserve inter-device privacy.…”

Section: Introductionmentioning

confidence: 99%

Personalized Decentralized Federated Learning with Knowledge Distillation

Jeong¹,

Kountouris²

2023

Preprint

View full text Add to dashboard Cite

Personalization in federated learning (FL) functions as a coordinator for clients with high variance in data or behavior. Ensuring the convergence of these clients' models relies on how closely users collaborate with those with similar patterns or preferences. However, it is generally challenging to quantify similarity under limited knowledge about other users' models given to users in a decentralized network. To cope with this issue, we propose a personalized and fully decentralized FL algorithm, leveraging knowledge distillation techniques to empower each device so as to discern statistical distances between local models. Each client device can enhance its performance without sharing local data by estimating the similarity between two intermediate outputs from feeding local samples as in knowledge distillation. Our empirical studies demonstrate that the proposed algorithm improves the test accuracy of clients in fewer iterations under highly non-independent and identically distributed (non-i.i.d.) data distributions and is beneficial to agents with small datasets, even without the need for a central server.

show abstract

Section: Introductionmentioning

confidence: 99%

Personalized Decentralized Federated Learning with Knowledge Distillation

Jeong¹,

Kountouris²

2023

Preprint

View full text Add to dashboard Cite

show abstract

“…In particular, MEC brings the high computing servers closer to the mobile users so that users with low computing and energy capability are able to offload their latency and computing-intensive tasks to the edge server [13]- [15]. Therefore, mobile users are able to offload a selected portion of local dataset to the edge server where a statistical model is trained by the edge server simultaneously with several mobile devices in hands [16], [17]. Even though FL is intended for the privacy preserving application, a portion of local dataset which are not privacysensitive can be offloaded to the MEC for further computation.…”

Section: Introductionmentioning

confidence: 99%

“…Inline with this idea, the works in [17] and [16] proposed the local data sharing mechanism for FL. In [16], the authors mitigated the non-i.i.d.…”

Section: Introductionmentioning

confidence: 99%

“…data problem by allowing a limited number of users to upload their local data to a server; and thus, the server trains a model on the uploaded data to support the FL process during model aggregation. Authors in [17] proposed a distributed data augmentation algorithm in which users share a fraction of their local dataset to confront the lack of on-device data samples. Similar to these approaches [16], [17], we give users the ultimate power to decide the dataset offloading.…”

Section: Introductionmentioning

confidence: 99%

“…Authors in [17] proposed a distributed data augmentation algorithm in which users share a fraction of their local dataset to confront the lack of on-device data samples. Similar to these approaches [16], [17], we give users the ultimate power to decide the dataset offloading. On the other hand, as a ML developer, the global model gets benefited with our proposed scheme, wherein we balance between the high accuracy obtained in a centralized setting and the distributed privacy preserving model training framework, such as FL.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Energy-Aware Resource Management for Federated Learning in Multi-Access Edge Computing Systems

et al. 2021

View full text Add to dashboard Cite

In Federated Learning (FL), a global statistical model is developed by encouraging mobile users to perform the model training on their local data and aggregating the output local model parameters in an iterative manner. However, due to limited energy and computation capability at the mobile devices, the performance of the model training is always at stake to meet the objective of local energy minimization. In this regard, Multi-access Edge Computing (MEC)-enabled FL addresses the tradeoff between the model performance and the energy consumption of the mobile devices by allowing users to offload a portion of their local dataset to an edge server for the model training. Since the edge server has high computation capability, the time consumption of the model training at the edge server is insignificant. However, the time consumption for dataset offloading from mobile users to the edge server has a significant impact on the total time consumed to complete a single round of FL process. Thus, resource management in MEC-enabled FL is challenging, where the objective is to reduce the total time consumption while saving the energy consumption of the mobile devices. In this article, we formulate an energy-aware resource management for MEC-enabled FL in which the model training loss and the total time consumption are jointly minimized, while considering the energy limitation of mobile devices. In addition, we recast the formulated problem as a Generalized Nash Equilibrium Problem (GNEP) to capture the coupling constraints between the radio resource management and dataset offloading. To that end, we analyze the impact of the dataset offloading and computing resource allocation on the model training loss, time, and the energy consumption. Finally, we present the convergence analysis of the proposed solution, and evaluate its performance against the traditional FL approach. Simulation results demonstrate the efficacy of our proposed solution approach.INDEX TERMS Dataset offloading, energy-aware resource management, federated learning, generalized Nash equilibrium game, multi-access edge computing.

show abstract

pvFed: Personalized Vertical Federated learning for Client‐Specific Tasks

Nishikawa,

Yanabe,

Sakuma

et al. 2024

IEEJ Transactions Elec Engng

View full text Add to dashboard Cite

Federated Learning (FL) is a distributed machine learning paradigm that enables multiple data holders to collaborate on building machine learning models while preserving the privacy of their data. FL can be categorized as horizontal or vertical, depending on the distribution characteristics of the data. Specifically, horizontal FL uses data partitioned in the sample space, whereas vertical FL uses data partitioned in the feature space. Traditional vertical FL methods aim to facilitate collaboration among clients to infer a single global target. However, these methods may be impractical because each client often has a unique target to be inferred. In this paper, we propose a novel vertical FL method, called personalized vertical federated learning (pvFed), which addresses this limitation by allowing each client to perform inferences specific to their individual task. To the best of our knowledge, no existing method currently addresses this limitation. The objective of pvFed is to construct a global model that generates a representation vector to support client inference. The global model, constructed using distillation and dimensionality reduction, takes a sample ID common to all clients as input and outputs a sample‐specific representation vector. Clients utilize the intermediate representation of their own model and the representation vectors output by the global model for inference. Because these vectors are not dependent on client‐specific tasks, clients can repurpose them for any additional tasks. Our experiments, conducted on two distinct data types—image and tabular data sets, under a vertical partitioning where each client had its own specific task, demonstrated the efficacy of vectors generated by the global model in pvFed for client inference. © 2024 Institute of Electrical Engineer of Japan and Wiley Periodicals LLC.

show abstract

Hiding in the Crowd: Federated Data Augmentation for On-Device Learning

Cited by 19 publications

References 8 publications

Personalized Decentralized Federated Learning with Knowledge Distillation

Personalized Decentralized Federated Learning with Knowledge Distillation

Energy-Aware Resource Management for Federated Learning in Multi-Access Edge Computing Systems

pvFed: Personalized Vertical Federated learning for Client‐Specific Tasks

Contact Info

Product

Resources

About