BlobSeer

Abstract

To accommodate the needs of large-scale distributed P2P systems, scalable data management strategies are required, allowing applications to efficiently cope with continuously growing, highly distributed data. This paper addresses the problem of efficiently storing and accessing very large binary data objects (blobs). It proposes an efficient versioning scheme allowing a large number of clients to concurrently read, write and append data to huge blobs that are fragmented and distributed at a very large scale. Scalability under heavy concurrency is achieved thanks to an original metadata scheme, based on a distributed segment tree built on top of a Distributed Hash Table (DHT). Our approach has been implemented and experimented within our BlobSeer prototype on the Grid'5000 testbed, using up to 175 nodes.

Adaptive Distributed Data Storage for Context-Aware Applications

“…This cannot support well a typical scenario envisioning millions of concurrent users writing their context data. BlobSeer [16] is a large-scale, distributed, binary storage service. It keeps versions for all records, so that concurrent read/write accesses are facilitated without affecting the high throughput of the system.…”

“…Clients can write new information in a chunk while others can read the old information, without needing to synchronize. Thus, BlobSeer [16] offers an appropriate alternative, as it provides real-time guarantees, large concurrent access guarantees, and support for eventual consistency through an advanced versioning mechanism. This is what motivated our choices in what follows.…”

“…For the Persistence layer we use the BlobSeer [16] storage distributed system. BlobSeer consists of a set of distributed entities that cooperate to enable a high throughput storage.…”