Scheduling memory access on a distributed cloud storage network

Rojas‐Cessa, Roberto; Cai, Lin X.; Kijkanjanarat, Taweesak

doi:10.1109/wocc.2012.6198152

Cited by 5 publications

(6 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Therefore, each chunk is placed on a different node to provide high reliability in the event of node or network failures. While data locality and network delay have been one of the key issues studied in data center scheduling algorithms [18], [19], [21], the prior work does not apply to erasure-coded systems.…”

Section: System Modelmentioning

confidence: 99%

“…To deal with them, we select a fixed reference point and leverage a linear approximation of (20) with in a small neighbourhood of the reference point. For all , we have (21) where is a sufficiently large constant relating to the approximation ratio. It is easy to see that the approximation approaches the real cost function within a small neighbourhood of as increases.…”

Section: B Constructing Convex Approximationsmentioning

confidence: 99%

“…Remark: To prove Theorem 2, we show that Algorithm JLCM generates a series of decreasing objective values of Problem JLCM with a modified cost function (24) The key idea in our proof is that the linear approximation of storage cost function in (21) can be seen as a subgradient of , which converges to the real storage cost function as , i.e.,…”

Section: Algorithm Jlcm and Convergence Analysismentioning

confidence: 99%

“…Thus, minimizing can be viewed as optimizing the difference between 2 convex objectives, namely and , which can be also solved via a Difference-of-Convex Programming (DCP). In this context, our linear approximation of cost function in (21) can be viewed as an approximated supergradient in DCP. Please refer to [39] for a comprehensive study of regularization techniques in DCP to speed up the convergence of Algorithm JLCM.…”

Section: Algorithm Jlcm and Convergence Analysismentioning

confidence: 99%

See 3 more Smart Citations

Joint Latency and Cost Optimization for Erasure-Coded Data Center Storage

Xiang

Lan

Aggarwal

et al. 2016

IEEE/ACM Trans. Networking

114

View full text Add to dashboard Cite

Modern distributed storage systems offer large capacity to satisfy the exponentially increasing need of storage space. They often use erasure codes to protect against disk and node failures to increase reliability, while trying to meet the latency requirements of the applications and clients. This paper provides an insightful upper bound on the average service delay of such erasure-coded storage with arbitrary service time distribution and consisting of multiple heterogeneous files. Not only does the result supersede known delay bounds that only work for a single file or homogeneous files, it also enables a novel problem of joint latency and storage cost minimization over three dimensions: selecting the erasure code, placement of encoded chunks, and optimizing scheduling policy. The problem is efficiently solved via the computation of a sequence of convex approximations with provable convergence. We further prototype our solution in an open-source cloud storage deployment over three geographically distributed data centers. Experimental results validate our theoretical delay analysis and show significant latency reduction, providing valuable insights into the proposed latency-cost tradeoff in erasure-coded storage.

show abstract

Section: System Modelmentioning

confidence: 99%

Section: B Constructing Convex Approximationsmentioning

confidence: 99%

Section: Algorithm Jlcm and Convergence Analysismentioning

confidence: 99%

Section: Algorithm Jlcm and Convergence Analysismentioning

confidence: 99%

See 2 more Smart Citations

Joint Latency and Cost Optimization for Erasure-Coded Data Center Storage

Xiang

Lan

Aggarwal

et al. 2016

IEEE/ACM Trans. Networking

114

View full text Add to dashboard Cite

show abstract

“…Therefore, each chunk is placed on a different node to provide high reliability in the event of node or network failures. While data locality and network delay have been one of the key issues studied in data center scheduling algorithms [19,20,22], the prior work does not apply to erasure-coded systems.…”

Section: Our Contributionsmentioning

confidence: 99%

Joint latency and cost optimization for erasurecoded data center storage

Xiang

Lan

Aggarwal

et al. 2014

SIGMETRICS Perform. Eval. Rev.

View full text Add to dashboard Cite

Abstract-Modern distributed storage systems offer large capacity to satisfy the exponentially increasing need of storage space. They often use erasure codes to protect against disk and node failures to increase reliability, while trying to meet the latency requirements of the applications and clients. This paper provides an insightful upper bound on the average service delay of such erasure-coded storage with arbitrary service time distribution and consisting of multiple heterogeneous files. Not only does the result supersede known delay bounds that only work for a single file or homogeneous files, it also enables a novel problem of joint latency and storage cost minimization over three dimensions: selecting the erasure code, placement of encoded chunks, and optimizing scheduling policy. The problem is efficiently solved via the computation of a sequence of convex approximations with provable convergence. We further prototype our solution in an open-source, cloud storage deployment over three geographically distributed data centers. Experimental results validate our theoretical delay analysis and show significant latency reduction, providing valuable insights into the proposed latency-cost tradeoff in erasure-coded storage.

show abstract