In‐network block repairing for erasure coding storage systems

Abstract

Summary In the erasure coding storage system, it is necessary to extract multiple data blocks from other remaining storage nodes to a new node when a storage node fails, which repairs the failed data block satisfactorily. However, this would incur the incast problem at this new node. The existing solutions for the repair process in the incast problem mainly rely on path planning and resource allocation. Although these solutions improve the performance of repairing the failed data blocks, they still waste a large amount of storage and bandwidth resources unavoidably. In this paper, we propose the incast problem to be resolved economically via the in‐network aggregation. Specifically, we assume that the switches in data centers have certain data processing capabilities and can aggregate data flows efficiently. Thereafter, we propose a set of in‐network methods to repair a failed data block in the erasure coding storage systems, taking the fat‐tree data center as an example. Thus, the incast problem can be solved effectively during the data transmission process. Compared with the prior methods, our approach effectively avoids the overhead of extra path computing, as well as significantly reduces the link cost of repairing data blocks, while promising similar or faster repair speed.

Cloud computing–based big data processing and intelligent analytics

“…Current solutions mainly rely on path planning and resource allocation and waste a large amount of storage and bandwidth resources unavoidably. Xia et al propose the incast problem to be resolved economically via the in‐network aggregation, where a set of in‐network methods to repair a failed data block in the erasure coding storage systems. Compared with the existing methods, the in‐network methods are capable of reducing the bandwidth consumption as well as achieving higher repair speed.…”