Long MDS codes for optimal repair bandwidth

Abstract

Abstract-MDS codes are erasure-correcting codes that can correct the maximum number of erasures given the number of redundancy or parity symbols. If an MDS code has r parities and no more than r erasures occur, then by transmitting all the remaining data in the code one can recover the original information. However, it was shown that in order to recover a single symbol erasure, only a fraction of 1/r of the information needs to be transmitted. This fraction is called the repair bandwidth (fraction). Explicit code constructions were given in previous works. If we view each symbol in the code as a vector or a column, then the code forms a 2D array and such codes are especially widely used in storage systems. In this paper, we ask the following question: given the length of the column l, can we construct high-rate MDS array codes with optimal repair bandwidth of 1/r, whose code length is as long as possible? In this paper, we give code constructions such that the code length is (r + 1) log r l.I. INTRODUCTION MDS (maximum distance separable) codes are optimal error-correcting codes in the sense that they have the largest minimum distance given the number of parity symbols. If each symbol is a vector or a column, we call such a code an MDS array code (e.g. . In (distributed) storage systems, each column is usually stored in a different disk, and MDS array codes are widely used to protect data against erasures due to their error correction ability and low computational complexity. In this paper, we call each symbol a column or a node, and the column length, or the vector size of a symbol, is denoted by l.If an MDS code has r parities, then it can correct up to r erasures of entire columns. In this paper, we not only would like to recover any e erasures, e ≤ r, but also care about the efficiency in recovery: what is the fraction of the remaining data transmitted in order to correct e erasures? We call this fraction the repair bandwidth (fraction). For example, if e = r erasures happen, it is obvious that we have to transmit all of the remaining information, therefore, the fraction is 1. For e = 1 erasure it was shown in [8] (which also formulated the repair problem) that this fraction is actually lowered bounded by 1/r. If e ≤ r symbols are erased and we repair them exactly as they were, this fraction is lower bounded by e/r [17]. If this bound is achieved for some code, we say it has optimal repair. Since the repair of information is much more crucial than redundancy, and we study mainly high-rate codes, we will focus on the optimal repair of information or systematic nodes. Moreover, since single erasure is the most common scenario in practice, we assume e = 1. For example, in Figure

Progress on high-rate MSR codes: Enabling arbitrary number of helper nodes

“…This construction enables repair of all the nodes in the system and works for d = n − 1, i.e., all the remaining n−1 nodes has to be contacted to repair a single failed node. The construction with polynomial sub-packetization and enabling repair of only systematic nodes are also presented in [13], [14]. As for the converse results, Goparaju et al establish a lower bound on the sub-packetization level of an MSR code with given n and k in [15].…”

MDS Code Constructions with Small Sub-packetization and Near-optimal Repair Bandwidth

“…Towards constructing high-rate exact-repairable MDS codes with optimal repair-bandwidth, Cadambe et al [3] show the existence of such codes when subpacketization level approaches infinity. Motivated by this result, the problem of designing high-rate exactrepairable MDS codes with finite sub-packetization level and optimal repair bandwidth is explored in [2,8,17,22,23,28,32,34,36] and references therein.…”

Access Versus Bandwidth in Codes for Storage

“…The lower bound is given by the code constructed in [20]. As one can notice, there exists a big gap between the upper and the lower bound.…”

“…The lower bound is derived by the codes constructed in [4], [20]. Note that [20] constructed also an optimal bandwidth code with k = (r + 1) log r l. Therefore, in the general case where we do not require an optimal update code, there is a difference between optimal access and optimal bandwidth code.…”

“…Note that [20] constructed also an optimal bandwidth code with k = (r + 1) log r l. Therefore, in the general case where we do not require an optimal update code, there is a difference between optimal access and optimal bandwidth code. Namely, these two properties are not equivalent (see Table 2).…”