Coding for Distributed Fog Computing in Internet of Mobile Things

Yue, Jing; Xiao, Ming

doi:10.1109/tmc.2019.2963668

Cited by 19 publications

(12 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Meanwhile, it is inefficient (sometimes even infeasible) to transmit all data to a central node for analysis. For the reason, distributed machine learning (DML), which stores and processes all or parts of data in different nodes, has attracted significant research interests and applications [ 1 , 3 , 4 , 5 , 6 , 7 , 8 , 9 , 10 , 11 , 12 , 13 , 14 , 15 , 16 ]. There are different methods of implementing DML, i.e., primal method (e.g., distributed gradient descend [ 4 , 7 ], federated learning [ 5 , 6 ]) and primal–dual method (e.g., alternating direction method of multipliers (ADMM)) [ 16 ].…”

Section: Background and Motivationsmentioning

confidence: 99%

Coding for Large-Scale Distributed Machine Learning

Xiao

Skoglund

2022

Entropy

Self Cite

View full text Add to dashboard Cite

This article aims to give a comprehensive and rigorous review of the principles and recent development of coding for large-scale distributed machine learning (DML). With increasing data volumes and the pervasive deployment of sensors and computing machines, machine learning has become more distributed. Moreover, the involved computing nodes and data volumes for learning tasks have also increased significantly. For large-scale distributed learning systems, significant challenges have appeared in terms of delay, errors, efficiency, etc. To address the problems, various error-control or performance-boosting schemes have been proposed recently for different aspects, such as the duplication of computing nodes. More recently, error-control coding has been investigated for DML to improve reliability and efficiency. The benefits of coding for DML include high-efficiency, low complexity, etc. Despite the benefits and recent progress, however, there is still a lack of comprehensive survey on this topic, especially for large-scale learning. This paper seeks to introduce the theories and algorithms of coding for DML. For primal-based DML schemes, we first discuss the gradient coding with the optimal code distance. Then, we introduce random coding for gradient-based DML. For primal–dual-based DML, i.e., ADMM (alternating direction method of multipliers), we propose a separate coding method for two steps of distributed optimization. Then coding schemes for different steps are discussed. Finally, a few potential directions for future works are also given.

show abstract

Section: Background and Motivationsmentioning

confidence: 99%

Coding for Large-Scale Distributed Machine Learning

Xiao

Skoglund

2022

Entropy

Self Cite

View full text Add to dashboard Cite

show abstract

“…The action performed by the agent is to allocate network resources for the end user function request based on the physical node mapping probability and the Floyd algorithm. We use the joint optimization result calculated by formula (14) as the agent's reward. Then we use the gradient descent method to train the neural network, the gradient calculation method is,…”

Section: Training and Testingmentioning

confidence: 99%

“…The edge server can serve end users in the region, and the system tasks can be completed through message passing coordination between edge servers. Taking into account the characteristics of the extensive deployment of SAGIN, it will be extremely necessary to adopt a distributed management solution for SAGIN [14].…”

Section: Introductionmentioning

confidence: 99%

Incorporating Distributed DRL into Storage Resource Optimization of Space-Air-Ground Integrated Wireless Communication Network

Wang

Liu

Jiang

et al. 2022

Preprint

View full text Add to dashboard Cite

“…One major challenge is that many computing frameworks are vulnerable to uncertain disturbances, such as node/link failures, communication congestion, and slow-downs [6]. Such disturbances, which can be modeled as stragglers that are slow or even fail in returning results, have been observed in many largescale computing systems such as cloud computing [7], mobile edge computing [8], and fog computing [9].…”

Section: Introductionmentioning

confidence: 99%

On Batch-Processing Based Coded Computing for Heterogeneous Distributed Computing Systems

Wang

Xie

et al. 2021

IEEE Trans. Netw. Sci. Eng.

View full text Add to dashboard Cite

In recent years, coded distributed computing (CDC) has attracted significant attention, because it can efficiently facilitate many delay-sensitive computation tasks against unexpected latencies in distributed computing systems. Despite such a salient feature, many design challenges and opportunities remain. In this paper, we focus on practical computing systems with heterogeneous computing resources, and design a novel CDC approach, called batch-processing based coded computing (BPCC), which exploits the fact that every computing node can obtain some coded results before it completes the whole task. To this end, we first describe the main idea of the BPCC framework, and then formulate an optimization problem for BPCC to minimize the task completion time by configuring the computation load. Through formal theoretical analyses, extensive simulation studies, and comprehensive real experiments on the Amazon EC2 computing clusters, we demonstrate promising performance of the proposed BPCC scheme, in terms of high computational efficiency and robustness to uncertain disturbances.

show abstract

Coding for Distributed Fog Computing in Internet of Mobile Things

Cited by 19 publications

References 35 publications

Coding for Large-Scale Distributed Machine Learning

Coding for Large-Scale Distributed Machine Learning

Incorporating Distributed DRL into Storage Resource Optimization of Space-Air-Ground Integrated Wireless Communication Network

On Batch-Processing Based Coded Computing for Heterogeneous Distributed Computing Systems

Contact Info

Product

Resources

About