Optimal locally repairable codes and connections to matroid theory

Abstract

Petabyte-scale distributed storage systems are currently transitioning to erasure codes to achieve higher storage efficiency. Classical codes like Reed-Solomon are highly sub-optimal for distributed environments due to their high overhead in single-failure events. Locally Repairable Codes (LRCs) form a new family of codes that are repair efficient. In particular, LRCs minimize the number of nodes participating in single node repairs during which they generate small network traffic. Two large-scale distributed storage systems have already implemented different types of LRCs: Windows Azure Storage and the Hadoop Distributed File System RAID used by Facebook. The fundamental bounds for LRCs, namely the best possible distance for a given code locality, were recently discovered, but few explicit constructions exist. In this work, we present an explicit and optimal LRCs that are simple to construct. Our construction is based on grouping Reed-Solomon (RS) coded symbols to obtain RS coded symbols over a larger finite field. We then partition these RS symbols in small groups, and re-encode them using a simple local code that offers low repair locality. For the analysis of the optimality of the code, we derive a new result on the matroid represented by the code's generator matrix.

Batch codes through dense graphs without short cycles

“…This was re-cently further generalized for nonlinear codes [25] and several explicit constructions of locally repairable codes have been investigated (e.g. [18,24,30,37,38]). Note that locally repairable codes are related but fundamentally different from locally decodable codes [42].…”

A connection between locally repairable codes and exact regenerating codes

“…Locally repairable codes for several parameter sets (N, K, D, R, ∆) are constructed in [6], [9], [21], [22], [11], [23]. In [6] it was proved that bound (2) is not achievable for all parameters (N, K, R).…”

Two-layer locally repairable codes for distributed storage systems

“…Various codes have been proposed to reduce repair locality, such as scalar linear codes [6] [8] and vector codes [7] [9] [10]. In [6] [8], extra parity constraints are introduced into encoded symbols of an MDS code to enhance repair locality, and a trade-off is demonstrated between the minimum distance and the repair locality of the resulting code.…”

“…In [6] [8], extra parity constraints are introduced into encoded symbols of an MDS code to enhance repair locality, and a trade-off is demonstrated between the minimum distance and the repair locality of the resulting code. The trade-off is extended in [7] to accommodate vector codes for local repair This work was supported in part by NSF under Grant ECCS-1055877.…”

“…An outer MDS code and an inner fractional repetition code are employed in [13] to construct regenerating codes with local, exact and uncoded repair. Scalar linear codes such as Pyramid codes investigated in [6] [14] and that proposed in [8] can also be viewed as examples of two-layer encoding.…”