Squeezed Polynomial Codes: Communication-Efficient Coded Computation in Straggler-Exploiting Distributed Matrix Multiplication

Abstract

In a distributed computing environment, there may exist slow processing workers, which are known as "stragglers", and they can slow down the whole computing process. In this paper, we consider coded computation for matrix multiplication tasks in distributed computing, which can mitigate the effect of stragglers by a coding approach. We propose a new communication-efficient coded computation scheme, namely squeezed polynomial codes, for a straggler-exploiting scenario where multiple sub-tasks are assigned to the workers to partially leverage the computation capability of stragglers. The key idea of squeezed polynomial codes is to overlap the encoded matrices in assigning multiple sub-tasks with appropriate polynomial functions in order to reduce the task-allocation communication load from a master to its workers. We compare squeezed polynomial codes with the existing schemes for distributed matrix multiplication in a communication load perspective. Consequently, we show that squeezed polynomial codes can efficiently reduce the communication load while ensuring the optimal recovery condition at a master to obtain final product. INDEX TERMS distributed computing, coded computation, matrix multiplication, polynomial codes.

Local Re-Encoding for Coded Matrix Multiplication

“…In order to make use of more resources efficiently, the coding schemes that leverage partial stragglers have also been proposed recently [25], [26]. On the other hand, other resources such as the bandwidth may also become a bottleneck, and communication-efficient coding schemes for distributed matrix multiplication have been proposed, e.g., squeezed polynomial codes that introduce replicated coded matrices [10] and local error-correcting codes in serverless systems [5]. However, the existing works above have only considered one kind of resource with a fixed coding scheme and parameters, while in practice the performances of different resources may vary with time.…”

“…Here, after re-encoding, we generate Ã′′ (x) which is encoded by a (λ m m, n, p) EP code from a matrix A ′′ with rows in A switched as in (10).…”

“…On the other hand, coding-based approaches for distributed matrix multiplication have been proposed to tolerate stragglers more efficiently [3]- [10], where each server multiplies coded matrices encoded from submatrices in A or/and B. Using the same example above, we still partition A and B into two submatrices, and encode them into (A 0 + A 1 ) and (B 0 + B 1 ), respectively.…”

Securely Straggler-Exploiting Coded Computation for Distributed Matrix Multiplication

“…Without loss of generality, we prove Lemma 3 for W E , E ⊂ [1 : W ], |E| = E, which can be generalized to any E workers. We use the simplified notations defined in (16) and the additional notation as…”

“…• In the proposed SSEC, the communication load for allocating sub-tasks to workers can be significantly reduced. The key idea of SSEC is to overlap encoded submatrices for assigning multiple sub-tasks with appropriate polynomial functions, as in [16], [17].…”