Efficient Encoding Schedules for XOR-Based Erasure Codes

Luo, Jianqiang; Shrestha, Mochan; Xu, Lizhong; Plank, James S.

doi:10.1109/tc.2013.23

Cited by 24 publications

(29 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…One improves the performance by reducing calculations in erasure coding [32], [33]. Some make efficient utilization of CPU hardware to accelerate erasure coding [21], [34]. The rest use parallel computing techniques to speed up erasure coding on modern acceleration hardware: CPUs and GPUs.…”

Section: ) Optimization On Erasure Codingmentioning

confidence: 99%

“…The main ways of accelerating erasure coding with CPU are through reducing cache misses and vectorization. A study [34] presents how data are loaded to CPU and analyzes the cache miss rate of different ways of loading data to CPU. It increases the spatial data locality to improve the cache hit rate to speed-up the performance of erasure coding.…”

Section: ) Optimization On Erasure Codingmentioning

confidence: 99%

See 1 more Smart Citation

An Erasure-Coded Storage System for Edge Computing

Liang

Zhao

et al. 2020

IEEE Access

View full text Add to dashboard Cite

Emerging computing paradigm edge computing expects to store and process data at the network edge with reduced latency and improved network bandwidth. To the best of our knowledge, key performance issues such as coding performance of erasure-coded storage systems haven't been investigated for edge computing. In this paper, we present an erasure-coded storage system for edge computing. Unlike the data center and cloud storage systems, it employs edge devices to perform encoding and decoding operations, which can be a performance bottleneck of the whole storage system due to limited computing power. Hence, we present a comprehensive study of the performance of erasure coding to see if it can match the network performance of 5G and Wi-Fi 6 at the network edge. We use the popular edge device Jetson Nano and two state-of-the-art coding libraries: Jerasure and G-CRS. Our evaluation results reveal unsatisfied performance for Jerasure and high variance for G-CRS. To obtain better and stable performance, we accelerate erasure code with OpenMP on a multi-core CPU. Our work demonstrates our acceleration can bring stable performance and match the network bandwidth of 5G and Wi-Fi 6 for some commonly used cases. Besides, our work offers a better understanding of erasure-coded storage systems for edge computing and can be served as a reference to any further optimization for such kind of systems at the network edge.INDEX TERMS Erasure-coded storage system, edge computing, erasure coding, jetson nano.

show abstract

Section: ) Optimization On Erasure Codingmentioning

confidence: 99%

Section: ) Optimization On Erasure Codingmentioning

confidence: 99%

An Erasure-Coded Storage System for Edge Computing

Liang

Zhao

et al. 2020

IEEE Access

View full text Add to dashboard Cite

show abstract

“…This kind of complexity is studied for Maximum-Distance Separable (MDS) codes in [3]. Other work has been done to reduce redundant xor operations by applying scheduling [4].…”

Section: Introductionmentioning

confidence: 99%

Polynomial ring transforms for efficient XOR-based erasure coding

Detchart

Lacan

2017

2017 IEEE International Symposium on Information Theory (ISIT)

View full text Add to dashboard Cite

Abstract-The complexity of software implementations of MDS erasure codes mainly depends on the efficiency of the finite field operations implementation. In this paper, we propose a method to reduce the complexity of the finite field multiplication by using simple transforms between a field and a ring to perform the multiplication in a ring. We show that moving to a ring reduces the complexity of the operations. Then, we show that this construction allows the use of simple scheduling to reduce the number of operations.

show abstract

“…The later has the flexibility on coding parameters design and the redundancy under the scale of Galois field, but the exponential growth of computational complexity in Galois field make it inefficient, especially in large scale storage system. To deal with this problem, Luo [10,11] made some improvement on field operations, and Plank [12,13] on CPU instructions. However, it remains a hard work on large scale system.…”

Section: Introductionmentioning

confidence: 99%