On Batch-Processing Based Coded Computing for Heterogeneous Distributed Computing Systems

Wang, Baoqian; Xie, Junfei; Lu, Kejie; Wan, Yan; Fu, Shengli

doi:10.1109/tnse.2021.3095040

Cited by 23 publications

(5 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We capture in this section the various bounds of Y δ k,i−1 by a single unifying description under different iterative optimization strategies. We conclude from ( 39), ( 45), (46), and ( 49) that Y δ k,i−1 is bounded as follows…”

Section: Distributed Agents Under Stochastic Gradient Descent Methodsmentioning

confidence: 78%

Hybrid Distributed Optimization for Learning Over Networks With Heterogeneous Agents

Nassralla,

Akl,

Dawy

2023

IEEE Access

View full text Add to dashboard Cite

This paper considers distributed optimization for learning problems over networks with heterogeneous agents having different computational capabilities. The heterogeneity of computational capabilities implies that a subset of the agents may run computationally-intensive learning algorithms like Newton's method or full gradient descent, while the other agents can only run lower-complexity algorithms like stochastic gradient descent. This leads to opportunities for designing hybrid distributed optimization algorithms that rely on cooperation among the network agents in order to enhance overall performance, improve the rate of convergence, and reduce the communication overhead. We show in this work that hybrid learning with cooperation among heterogeneous agents attains a stable solution. For small step-sizes µ, the proposed approach leads to small estimation error in the order of O(µ). We also provide the theoretical analysis of the stability of the first, second, and fourth order error moments for learning over networks with heterogeneous agents. Finally, results are presented and analyzed for case study scenarios to demonstrate the effectiveness of the proposed approach.

show abstract

Section: Distributed Agents Under Stochastic Gradient Descent Methodsmentioning

confidence: 78%

Hybrid Distributed Optimization for Learning Over Networks With Heterogeneous Agents

Nassralla,

Akl,

Dawy

2023

IEEE Access

View full text Add to dashboard Cite

show abstract

“…Further, the scheme allows the enhancement in the efficiency by the dynamic loading of the object sharing for the task. Batch processing-based coded distributed computing has been incorporated in [81] to issue of latency in the distributed computing network. The optimization problem is based on the minimization of the task completion time to configure the computation load [81].…”

Section: A Related Workmentioning

confidence: 99%

“…Batch processing-based coded distributed computing has been incorporated in [81] to issue of latency in the distributed computing network. The optimization problem is based on the minimization of the task completion time to configure the computation load [81]. A coding theoretic scheme is investigated to handle the straggler-like problem.…”

Section: A Related Workmentioning

confidence: 99%

6G: Technology Evolution in Future Wireless Networks

Shafi,

Jha,

Jain

2024

IEEE Access

View full text Add to dashboard Cite

The Sixth Generation (6G) Wireless Communication Network (WCN) is the successive provision to ameliorate the gain with ultra-low latency, and e xtremely high energy efficiency. The 6G WCN enables the specifications of artificial intelligence to optimize the services and capabilities. The vision of the 6G era is expected to address a seamless fusion of communication between the human, physical world, and digital world. The latest 6G WCN standard is a fundamental foundation and requires immense attention in the field of research. This paper presents the framework of 6G WCN with an illustration of its key technologies. The different technologies involved in 6G are well explained with the demonstration of the communication scenario such that the key performance indicators are improved with major differences. The primary contribution of this paper is the explanation of the 6G with the technologies that have a drastic impact on the characteristic aspects of a wireless communication network such as data rate, spectrum efficiency, energy efficiency, connection density, and reliability. All these technologies have the capability to revolutionize the subsequent WCN.

show abstract

“…Proof. Based on Lemma 2 and the characteristics of order statistic, L cmp can be obtained by taking the expectation of (8).…”

Section: Proof See Appendix Bmentioning

confidence: 99%

“…It implied that MDS coding scheme could reduce the computation time significantly and achieve an order-wise improvement over the original uncoded scheme. The authors in [7], [8] further studied the corresponding scheme to allocate the optimal computational task for nodes in the heterogeneous networks with disparate computation capabilities.…”

mentioning

confidence: 99%

Wireless Coded Computation With Error Detection

Fang,

Chen,

Chen

et al. 2024

IEEE Trans. Commun.

View full text Add to dashboard Cite

In wireless networks with distributed computing, the computational performance is limited by stragglers. To mitigate the stragglers' effect, coded computation is adopted through computational redundancy. Moreover, in wireless transmission, transmission errors may occur due to noise, channel fading and so on. Existing works design coded computation and error detection separately. However, this leads to frequent encoding and inefficient allocation. In this paper, we propose a joint computation and transmission coding (JCTC) scheme to design coded computation and error detection jointly. The coded computation is based on Luby transform (LT) code and linear error-detecting codes are applied for the re-transmission mechanism. To achieve the low dynamic encoding, two-layer encoding is adopted. Then, the performances of JCTC scheme are analyzed in terms of latency and computation reliability. Finally, in order to achieve efficient task and redundancy allocation, the wireless LT coded computation with error detection (WLTCC-ED) algorithm is given from both iterative and low-complexity perspectives respectively. Through theoretical analysis and numerical simulation, it shows that our proposed JCTC scheme has significant advantages over separate designs.

show abstract

On Batch-Processing Based Coded Computing for Heterogeneous Distributed Computing Systems

Cited by 23 publications

References 33 publications

Hybrid Distributed Optimization for Learning Over Networks With Heterogeneous Agents

Hybrid Distributed Optimization for Learning Over Networks With Heterogeneous Agents

6G: Technology Evolution in Future Wireless Networks

Wireless Coded Computation With Error Detection

Contact Info

Product

Resources

About