Heterogeneous Coded Distributed Computing: Joint Design of File Allocation and Function Assignment

Abstract

This paper studies the computation-communication tradeoff in a heterogeneous MapReduce computing system where each distributed node is equipped with different computation capability. We first obtain an achievable communication load for any given computation load and any given function assignment at each node. The proposed file allocation strategy has two steps: first, the input files are partitioned into disjoint batches, each with possibly different size and computed by a distinct node; then, each node computes additional files from its non-computed files according to its redundant computation capability. In the Shuffle phase, coded multicasting opportunities are exploited thanks to the repetitive file allocation among different nodes. Based on this scheme, we further propose the computation-aware and the shuffle-aware function assignments. We prove that, by using proper function assignments, our achievable communication load for any given computation load is within a constant multiplicative gap to the optimum in an equivalent homogeneous system with the same average computation load. Numerical results show that our scheme with shuffle-aware function assignment achieves better computationcommunication tradeoff than existing works in some cases.coded multicasting opportunities are created as many as possible in the Shuffle phase to obtain the optimal computation-communication tradeoff. However, they only consider heterogeneous file allocation in the Map phase due to different storage size, and still assume homogeneous function assignment in the Reduce phase without taking the different computation capabilities across nodes into account. The authors in [13], [14] consider the heterogeneous systems where each node is assigned different number of output functions. Both works obtain an achievable communication load which is within a constant multiplicative gap to the optimum given the considered function assignment. They find that, by assigning more output functions to nodes with more input files, their proposed schemes even outperform the optimal scheme in an equivalent homogeneous system [3] in some cases. However, the heterogeneous systems considered in [13],[14] consist of multiple homogeneous systems where nodes in each system have the same storage and computation capabilities but differ from nodes in other systems, and is thus not suitable to

Cascaded Coded Distributed Computing on Heterogeneous Networks

“…work allows us to achieve a lower communication load that is less than L 1 , even for homogeneous networks. Similar observations were made for a heterogeneous network with s = 1 in [10], [11].…”

A New Combinatorial Design of Coded Distributed Computing

“…However, the KR design is not flexible because it requires K/r to be a positive integer. In [7], [15], the heterogeneous system was investigated. However, the optimal designs have not been empirically tested and in theory arXiv:2008.05631v1 [cs.IT] 13 Aug 2020 only work for a system with 3 nodes or have a complexity comparable to the LMYA design.…”

Leveraging User-Diversity in Energy-Efficient Edge-Facilitated Collaborative Fog Computing

“…The system model used in this paper is inspired by previous works on wireless distributed computing (WDC) [23]- [28]. Most notably, we also use the Map-Reduce distributed computing framework [29] with an access point (AP) or base station (BS) facilitating the communications between devices (i.e., communications are edge-facilitated).…”

“…The two approaches are however not mutually exclusive and could be combined in future collaborative-computing schemes to further improve the global energy-efficiency of the system. Most existing works on WDC consider the set of collaborating devices to be homogeneous in terms of computing and communication capabilities (with recent notable exceptions -still focused on CDC -like [24] and [28]). Under these conditions, the computing load is thus uniformly distributed across all devices.…”