Online Federated Multitask Learning

Li, Rui; Ma, Fenglong; Jiang, Wenjun; Gao, Jing

doi:10.1109/bigdata47090.2019.9006060

Cited by 27 publications

(14 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Second, the framework reschedules the training of clients to solve the local imbalance. In order to deal with a scenario where new devices keep joining the system, Li et al [22] introduce an online FL approach that derives model parameters for new devices based on the local data and existing model without revisiting the data of existing devices. The approach shows a comparable accuracy to conventional algorithms with smaller computation, communication and storage costs.…”

Section: A Learning Efficiency Of Flmentioning

confidence: 99%

Federated Learning for Vehicular Internet of Things: Recent Advances and Open Issues

Yoshinaga

et al. 2020

IEEE Open J. Comput. Soc.

284

106

View full text Add to dashboard Cite

show abstract

Section: A Learning Efficiency Of Flmentioning

confidence: 99%

Federated Learning for Vehicular Internet of Things: Recent Advances and Open Issues

Yoshinaga

et al. 2020

IEEE Open J. Comput. Soc.

284

106

View full text Add to dashboard Cite

show abstract

“…However, upon having extreme data heterogeneity across the devices, training a single model may result in poor model performance for a portion of devices. This has motivated a new trend of research in ML that aims to train userspecific ML models, referred to as multi-task learning [34], [35] or personalized federated learning [36], in the latter of which meta-gradient updates are introduced to enhance the training efficiency. As compared to this literature, we propose and investigate a new distributed ML paradigm over wireless networks called hierarchical nested personalized federated learning (HN-PFL) inspired by meta-gradient updates.…”

Section: Related Workmentioning

confidence: 99%

“…Constraint (34) imposed on the leader UAVs guarantees that there is enough energy remaining after parameter broadcasting and flying to reach the nearest recharging station. Constraints (35) and (36) ensure that the total amount of data offloaded from each device i ∈ C(u, s) and each coordinator UAV h ∈ W u is less than the size of the available data set. As a result of this offloading, (37) and (38) capture the total number of datapoints at the UAVs.…”

Section: Joint Energy and Performance Optimizationmentioning

confidence: 99%

UAV-assisted Online Machine Learning over Multi-Tiered Networks: A Hierarchical Nested Personalized Federated Learning Approach

Wang¹,

Hosseinalipour²,

Gorlatova³

et al. 2021

Preprint

View full text Add to dashboard Cite

We consider distributed machine learning (ML) through unmanned aerial vehicles (UAVs) for geo-distributed device clusters. We propose five new technologies/techniques: (i) stratified UAV swarms with leader, worker, and coordinator UAVs, (ii) hierarchical nested personalized federated learning (HN-PFL): a holistic distributed ML framework for personalized model training across the worker-leader-core network hierarchy, (iii) cooperative UAV resource pooling for distributed ML using the UAVs' local computational capabilities, (iv) aerial data caching and relaying for efficient data relaying to conduct ML, and (v) concept/model drift, capturing online data variations at the devices. We split the UAV-enabled model training problem as two parts. (a) Network-aware HN-PFL, where we optimize a tradeoff between energy consumption and ML model performance by configuring data offloading among devices-UAVs and UAV-UAVs, UAVs' CPU frequencies, and mini-batch sizes subject to communication/computation network heterogeneity. We tackle this optimization problem via the method of posynomial condensation and propose a distributed algorithm with a performance guarantee. (b) Macro-trajectory and learning duration design, which we formulate as a sequential decision making problem, tackled via deep reinforcement learning. Our simulations demonstrate the superiority of our methodology with regards to the distributed ML performance, the optimization of network resources, and the swarm trajectory efficiency.

show abstract

“…In [1,12], different multitask extensions of online learning are investigated, see also [3,17,6,13] for related extensions to meta-learning. Some online multitask applications are studied in [32,27,28], but without providing any regret analyses. In [33,40] authors extend the results of [10] to dynamically updated interaction matrices.…”

Section: Related Workmentioning

confidence: 99%

“…with L ∈ R N ×d and µ ∈ R N the Lagrange multipliers associated to constraints ( 29) and (28) respectively. For all i ≤ N , j ≤ d, differentiating with respect to Y ij and setting the gradient to 0 yields:…”

Section: A Technical Proofsmentioning

confidence: 99%

Multitask Online Mirror Descent

Cesa-Bianchi¹,

Laforgue²,

Paudice³

et al. 2021

Preprint

View full text Add to dashboard Cite

We introduce and analyze MT-OMD, a multitask generalization of Online Mirror Descent (OMD) which operates by sharing updates between tasks. We prove that the regret of MT-OMD is of order 1 + σ 2 (N − 1) √T , where σ 2 is the task variance according to the geometry induced by the regularizer, N is the number of tasks, and T is the time horizon. Whenever tasks are similar, that is, σ 2 ≤ 1, this improves upon the √ N T bound obtained by running independent OMDs on each task. Our multitask extensions of Online Gradient Descent and Exponentiated Gradient, two important instances of OMD, are shown to enjoy closed-form updates, making them easy to use in practice. Finally, we provide numerical experiments on four real-world datasets which support our theoretical findings.Preprint. Under review.

show abstract

Online Federated Multitask Learning

Cited by 27 publications

References 5 publications

Federated Learning for Vehicular Internet of Things: Recent Advances and Open Issues

Federated Learning for Vehicular Internet of Things: Recent Advances and Open Issues

UAV-assisted Online Machine Learning over Multi-Tiered Networks: A Hierarchical Nested Personalized Federated Learning Approach

Multitask Online Mirror Descent

Contact Info

Product

Resources

About