On the optimal recovery threshold of coded matrix multiplication

Fahim, Mohammad; Jeong, Haewon; Haddadpour, Farzin; Dutta, Sanghamitra; Cadambe, Viveck R.; Grover, Pulkit

doi:10.1109/allerton.2017.8262882

Cited by 74 publications

(127 citation statements)

References 16 publications

Supporting

Mentioning

125

Contrasting

Order By: Relevance

“…BACKGROUND: GENERALIZED POLYDOT CODE WITHOUT SECURITY CONSTRAINT In this section, we consider the system model shown in Fig. 1 and review the GPD construction first proposed in [15] and later improved in [14], [27] for the special case of no secrecy constrains, i.e., P C = 0. In the process, we propose a novel intuitive interpretation of GPD encoding and decoding based on the distributed computation of samples from convolutions via z-transforms.…”

Section: Private and Secure Matrix Multiplicationmentioning

confidence: 99%

“…We also derive the corresponding achievable set of triples (P C , P R , C L ). As we will discuss, the projection of this set onto the plane defined by the condition P C = 0 includes the set of pairs (P R , C L ) in (15) and (16) obtained by the GPD code [14]. The proposed secure GPD (SGPD) code augments matrices A and B by adding P C random block matrices to the input matrices A and B, in a manner similar to prior works [18]- [21], [23], yielding augmented matrices A * and B * .…”

Section: Secure Polydot Codementioning

confidence: 99%

See 1 more Smart Citation

Private and Secure Distributed Matrix Multiplication With Flexible Communication Load

Aliasgari

Simeone

Kliewer

2020

IEEE Trans.Inform.Forensic Secur.

View full text Add to dashboard Cite

Large matrix multiplications are central to largescale machine learning applications. These operations are often carried out on a distributed computing platform with a master server and multiple workers in the cloud operating in parallel. For such distributed platforms, it has been recently shown that coding over the input data matrices can reduce the computational delay, yielding a trade-off between recovery threshold, i.e., the number of workers required to recover the matrix product, and communication load, i.e., the total amount of data to be downloaded from the workers. In this paper, in addition to exact recovery requirements, we impose security and privacy constraints on the data matrices, and study the recovery threshold as a function of the communication load. We first assume that both matrices contain private information and that workers can collude to eavesdrop on the content of these data matrices. For this problem, we introduce a novel class of secure codes, referred to as secure generalized PolyDot (SGPD) codes, that generalize state-of-the-art non-secure codes for matrix multiplication. SGPD codes allow a flexible trade-off between recovery threshold and communication load for a fixed maximum number of colluding workers while providing perfect secrecy for the two data matrices. We then assume that one of the data matrices is taken from a public set known to all the workers. In this setup, the identity of the matrix of interest should be kept private from the workers. For this model, we present a variant of generalized PolyDot codes that can guarantee both secrecy of one matrix and privacy for the identity of the other matrix for the case of no colluding servers.

show abstract

Section: Private and Secure Matrix Multiplicationmentioning

confidence: 99%

Section: Secure Polydot Codementioning

confidence: 99%

Private and Secure Distributed Matrix Multiplication With Flexible Communication Load

Aliasgari

Simeone

Kliewer

2020

IEEE Trans.Inform.Forensic Secur.

View full text Add to dashboard Cite

show abstract

“…With memory per worker fixed, the recovery threshold of polynomial codes is further improved by MatDot codes [10], albeit at the cost of increased per-worker computation. In addition to MatDot coding, [10] introduces polyDot coding as a generalization and unification of polynomial and MatDot codes. PolyDot codes provide a tradeoff between the recovery threshold and the computation load assigned to each worker.…”

Section: A Background: Stragglers and Coded Computingmentioning

confidence: 99%

“…PolyDot codes [10] and entangled polynomial codes [11] slice the 3D cuboid along all (x-, y-, z-) axes. This is what we termed combinatorial partitioning (cf., Fig.2d).…”

Section: B 3d Visualization: Data Partitioning In Coded Computingmentioning

confidence: 99%

Hierarchical coded matrix multiplication

Kiani

Ferdinand

Draper

2019

2019 16th Canadian Workshop on Information Theory (CWIT)

View full text Add to dashboard Cite

In distributed computing systems slow working nodes, known as stragglers, can greatly extend finishing times. Coded computing is a technique that enables straggler-resistant computation. Most coded computing techniques presented to date provide robustness by ensuring that the time to finish depends only on a set of the fastest nodes. However, while stragglers do compute less work than non-stragglers, in real-world commercial cloud computing systems (e.g., Amazon's Elastic Compute Cloud (EC2)) the distinction is often a soft one. In this paper, we develop hierarchical coded computing that exploits the work completed by all nodes, both fast and slow, automatically integrating the potential contribution of each. We first present a conceptual framework to represent the division of work amongst nodes in coded matrix multiplication as a cuboid partitioning problem. This framework allows us to unify existing methods and motivates new techniques. We then develop three methods of hierarchical coded computing that we term bit-interleaved coded computation (BICC), multilevel coded computation (MLCC), and hybrid hierarchical coded computation (HHCC). In this paradigm, each worker is tasked with completing a sequence (a hierarchy) of ordered subtasks. The sequence of subtasks, and the complexity of each, is designed so that partial work completed by stragglers can be used in, rather than ignored. We note that our methods can be used in conjunction with any coded computing method. We illustrate this showing how we can use our methods to accelerate all previously developed coded computing technique by enabling them to exploit stragglers. Under a widely studied statistical model of completion time, our approach realizes a 66% improvement in expected finishing time. On Amazon EC2, the gain was 28% when stragglers are simulated.

show abstract

“…In particular, in [13], the code is designed such that the results of different workers form a maximum separable code (MDS), meaning that the final result can be recovered from any subset of servers with the minimum size. That approach has been extended to general matrix partitioning in [14], [15], and to the cases where only an approximate result of the matrix multiplication is needed [17], [20].…”

Section: Introductionmentioning

confidence: 99%

Limited-Sharing Multi-Party Computation for Massive Matrix Operations

Nodehi

Maddah-Ali

2018

2018 IEEE International Symposium on Information Theory (ISIT)

View full text Add to dashboard Cite

In this paper, we consider a secure multi-party computation problem (MPC), where the goal is to offload the computation of an arbitrary polynomial function of some massive private matrices to a network of workers. The workers are not reliable, some may collude to gain information about the input data. The system is initialized by sharing a (randomized) function of each input matrix to each server. Since the input matrices are massive, the size of each share is assumed to be at most 1/k fraction of the input matrix, for some k ∈ N. The objective is to minimize the number of workers needed to perform the computation task correctly, such that even if an arbitrary subset of t − 1 workers, for some t ∈ N, collude, they cannot gain any information about the input matrices. We propose a sharing scheme, called polynomial sharing, and show that it admits basic operations such as adding and multiplication of matrices, and transposing a matrix. By concatenating the procedures for basic operations, we show that any polynomial function of the input matrices can be calculated, subject to the problem constraints. We show that the proposed scheme can offer order-wise gain, in terms of number of workers needed, compared to the approaches formed by concatenation of job splitting and conventional MPC approaches.

show abstract

On the optimal recovery threshold of coded matrix multiplication

Cited by 74 publications

References 16 publications

Private and Secure Distributed Matrix Multiplication With Flexible Communication Load

Private and Secure Distributed Matrix Multiplication With Flexible Communication Load

Hierarchical coded matrix multiplication

Limited-Sharing Multi-Party Computation for Massive Matrix Operations

Contact Info

Product

Resources

About