DR-Update: A Dual-Level Relay Scheme in Erasure-Coded Storage Systems for Balanced Updates

Deng, Mingzhu; Yu, Songping; Xiao, Nong; Liu, Fang; He, Wen

doi:10.1109/dasc-picom-datacom-cyberscitec.2017.185

Cited by 1 publication

(7 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To deploy erasure coding in data centers, existing approaches [10], [24], [25], [33] mostly adopt hierarchical node placement by placing nodes on racks ( < ). We consider erasure-coded storage in a data center, and the bandwidth of its intra-rack and cross-rack links varies greatly.…”

Section: A Rs (N K) Code and Its Data Update Modementioning

confidence: 99%

“…We evaluate the update efficiency by the Update Time, which is defined as the total duration from the update request to the end of the update. We consider three standard configurations of ( , ) code in HDFS-RAID: (1) RS (9,6) [ =9, =6, =3] (2) RS (10,6) [ =10, =6, =4] (3) RS (14,10) [ =14, =10, =4] and test the update efficiency of the four schemes under different cross-rack bandwidths. In actual production, the intra-rack bandwidth is 10Gb/s, the cross-rack bandwidth is 1Gb/s [25].…”

Section: ) Update Efficiency Assessmentmentioning

confidence: 99%

“…Figure 10 describes the relationship between the update time and cross-rack bandwidth of the RS codes of the above three fault-tolerant parameters using RCW, RMW, SDCUP-R and SDCUP-I. Compared with RS (9,6) and RS (10,6), RS (9,6) using SDCUP-I at 1Gb/s has 31.3% less update time than RS (9,6) using RCW, RS (10,6) using SDCUP-I has 37.5% less update time than RS (10,6) using RCW. Because the increase in value lead to increase the data transmission.…”

Section: ) Update Efficiency Assessmentmentioning

confidence: 99%

“…However, the schemes in this paper are superior to other schemes because the data blocks are aggregated in the rack, which reduces the amount of data transferred. Comparing RS (10,6) and RS (14,10), at 0.5Gb/s, RS (10,6) uses SDCUP-I to reduce the update time by 35.5% compared to RCW. RS (14,10) uses SDCUP-I to reduce the update time by 47.2% compared to RCW.…”

Section: ) Update Efficiency Assessmentmentioning

confidence: 99%

“…Such tendency will have a great impact on system reliability and even cause permanent data loss. Therefore, the recent researches on erasure coding focus on mitigating data update overhead by proposing new codes with low update complexity [7], [8], data blocks layout strategy [9] and data update optimization algorithm [10]- [16].…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

SDCUP: Software-Defined-Control Based Erasure-Coded Collaborative Data Update Mechanism

Chen

Zhong

2020

IEEE Access

View full text Add to dashboard Cite

The storage system often uses erasure coding to provide the necessary fault tolerance. The erasure-coded update involves data transmission and data calculation of multiple nodes. Frequent updates will cause massive communication overhead. This paper mainly considers two issues: (1) In the scenario of frequent small-size updates, there are repetitive behaviors upon update, which causes bandwidth consumption to increase exponentially as the number of update nodes increases; (2) with the increase of data scale, there are local link busy phenomena caused by unbalanced use of links during update, which can prone to bottleneck links. In order to improve the inefficient update due to network bottlenecks. We propose SDCUP, a software-defined-control collaborative update mechanism that reduce update time for erasure-coded data with network load balance. Specially, SDCUP uses the software-defined control method to select the update transmission path according to the actual link load and adjust the data flow transmission rate by monitoring the degree of network load balance periodically. To further reduce the cross-rack update traffic, SDCUP unloads the calculation to the switch to realize the data aggregation in the rack, and it parallelizes sub-update operations to efficiently and cooperatively update. To evaluate the performance of SDCUP, we conduct simulation experiments on Mininet with real-world traces. The simulation results show that SDCUP can achieve a better load balance in multiple scenarios. Compared with the other data update schemes, the proposed method can improve the system throughput by up to 21% and reduce the update time by up to 47%. INDEX TERMS Erasure coding, software-defined, data update, network load balance.

show abstract