BlobSeer: Efficient data management for data-intensive applications distributed at large-scale

Nicolae, Bogdan; Antoniu, Gabriel; Bougé, Luc

doi:10.1109/ipdpsw.2010.5470802

Cited by 2 publications

(1 citation statement)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Finally, we implement and evaluate our approach through the BlobSeer [20] prototype. We performed a series of synthetic benchmarks that push the system to its limits and show a high throughput under heavy access concurrency, even for small granularity and under faults.…”

Section: Contributionmentioning

confidence: 99%

BlobSeer: Next-generation data management for large scale infrastructures

Nicolae

Antoniu

Bougé

et al. 2011

Journal of Parallel and Distributed Computing

Self Cite

107

View full text Add to dashboard Cite

As data volumes increase at a high speed in more and more application fields of science, engineering, information services, etc., the challenges posed by data-intensive computing gain an increasing importance. The emergence of highly scalable infrastructures, e.g. for cloud computing and for petascale computing and beyond introduces additional issues for which scalable data management becomes an immediate need. This paper brings several contributions. First, it proposes a set of principles for designing highly scalable distributed storage systems that are optimized for heavy data access concurrency. In particular, we highlight the potentially large benefits of using versioning in this context. Second, based on these principles, we propose a set of versioning algorithms, both for data and metadata, that enable a high throughput under concurrency. Finally, we implement and evaluate these algorithms in the BlobSeer prototype, that we integrate as a storage backend in the Hadoop MapReduce framework. We perform extensive microbenchmarks as well as experiments with real MapReduce applications: they demonstrate that applying the principles defended in our approach brings substantial benefits to data intensive applications.

show abstract

Section: Contributionmentioning

confidence: 99%