A Performance Evaluation of Erasure Coding Libraries for Cloud-Based Data Stores

Burihabwa, Dorian; Felber, Pascal; Mercier, Hugues; Schiavoni, Valerio

doi:10.1007/978-3-319-39577-7_13

Cited by 12 publications

(10 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In [15], an explicit structure of the first known system (n, k) MDS array code is described where n − k is equal to some constants so that the amount of information needed to reconstruct an erased column is equal to 1/(n − k), matching the lower bound of information theory. Recently, there has been considerable interest in incorporating secrets into erasure codes for distributed storage, such as [33]- [36]. These codes can also be considered as threshold secret sharing schemes.…”

Section: Introductionmentioning

confidence: 99%

Secure Secret Sharing With Adaptive Bandwidth in Distributed Cloud Storage Systems

Huagang

et al. 2020

IEEE Access

View full text Add to dashboard Cite

The development of cloud storage technology has brought us great convenience that we can store data in remote servers and access it on any connected device. However, the frequent leakage of private data has brought more and more attention to the protection of private data. To solve related problems, the distributed storage schemes have been proposed. Considering security and fault tolerance, many of those schemes adopt threshold secret sharing techniques which are wildly used in distributed storage systems with disaster tolerance function. Nevertheless, in practical situations, the bandwidth between users and different servers may be unbalanced or even unfixed, which leads to low communication efficiency of the schemes when adopting original secret sharing. To obtain higher communication efficiency under different communication loads, we proposed a novel adaptive bandwidth secret sharing scheme in distribution cloud storage systems. In addition, we consider the scenario in which we use Shamir's secret sharing scheme and Staircase codes respectively in order to improve the applicability of the adaptive bandwidth scheme. We make the comparative of performance analysis to show the advantages and disadvantages of these two schemes. In general, compared with the non-adaptive schemes, the proposed adaptive bandwidth scheme can make full use of unbalanced bandwidth and achieve higher average communication rate when the upper bound of bandwidth is large enough.

show abstract

Section: Introductionmentioning

confidence: 99%

Secure Secret Sharing With Adaptive Bandwidth in Distributed Cloud Storage Systems

Huagang

et al. 2020

IEEE Access

View full text Add to dashboard Cite

show abstract

“…From that point on, the days of a targeted document on the internet are counted. While popular content can rely on the Streisand effect 1 and alternative networks to keep on living a little longer, unpopular content can just disappear off the face of the internet. And when refusing to delete the content also exposes the platform or provider to cyber attacks [124], removing the content seems like the best option to avoid exposure but also reduce their storage costs.…”

Section: Context and Motivationmentioning

confidence: 99%

“…Finally, we look at how storage overhead can be tamed beyond with better compression gains than classic data reduction techniques without sacrificing on performance. [32,34,31] Chapter 4 First and fully functional implementation of STeP [17,141] Chapter 5 Implementation and evaluation of a framework for stackable user-space file systems [207,33] Chapter 6 Integration and evalution of a data deduplication scheme in user-space file system [183] 1.…”

Section: Taming Storage Overhead With Practical and Efficient Data Rementioning

confidence: 99%

“…• A benchmark of erasure coding libraries in a distributed setting [32]. This work was done under the supervision of Pascal Felber and Valerio Schiavoni with the collaboration of Hugues Mercier.…”

Section: Contributionsmentioning

confidence: 99%

“…The architecture is depicted in Figure 3 The proxy component acts as Playcloud's front-end and is responsible for keeping a mapping between client files and the actual storage backends where they are stored. 1 Clients, which run on independent nodes, contact the proxy component to write or read data through a simple REST interface that mimics the operating principles of well-established services like Amazon S3. The interactions between the proxy and the clients happen via synchronous HTTP messages over pre-established TCP channels.…”

Section: Architecturementioning

confidence: 99%

See 2 more Smart Citations

Erasure coding for distributed storage systems

Burihabwa¹

Self Cite

View full text Add to dashboard Cite

Cloud storage has durably entered the stage as go-to solution for business and personal storage. Virtually extending storage capabilities to infinity, cloud storage enables companies and individuals to focus on content creation without fear of running out of space or losing data. But as users entrust more and more data to the cloud, they also have to accept a loss of control over the data they o˜oad to the cloud. At a time when online services seem to be making a significant part of their profits by exploiting customer data, concerns over privacy and integrity of said data naturally arise. Are their online documents read by the storage provider or its employees? Is the content of these documents shared with third party partners of the storage provider? What happens if the provider goes bankrupt? Whatever answer can be o˙ered by the storage provider, the loss of control should be cause for concern. But storage providers also have to worry about trust and reliability. As they build distributed solutions to accommodate their customers’ needs, these concerns of control extend to the infrastructure they operate on. Conciliating security, confidentiality, resilience and perform-ance over large sets of distributed storage nodes is a tricky balancing act. And even when a suitable balance can be found, it is often done at the expense of increased storage overhead. In this dissertation, we try to mitigate these issues by focusing on three aspects. First, we study solutions to empower users with flexible tooling ensuring security, integrity and redundancy in distributed storage settings. By leveraging public cloud storage o˙erings to build a configurable file system and storage middleware, we show that securing cloud-storage from the client-side is an e˙ective way maintaining control. Second, we build a distributed archive whose resilience goes beyond standard redundancy schemes. To achieve this, we implement Recast, relying on a data entanglement scheme, that encodes and distributes data over a set of storage nodes to ensure durability at a manageable cost. Finally, we look into o˙setting the increase in storage overhead by means of data reduction. This is made possible by the use of Generalised Deduplication, a scheme that improves over classical data deduplication by detecting similarities beyond exact matches.

show abstract

ARES: Adaptive, Reconfigurable, Erasure Coded, Atomic Storage

Nicolaou

Cadambe

Prakash

et al. 2019

2019 IEEE 39th International Conference on Distributed Computing Systems (ICDCS)

View full text Add to dashboard Cite

Emulating a shared atomic, read/write storage system is a fundamental problem in distributed computing. Replicating atomic objects among a set of data hosts was the norm for traditional implementations (e.g., [11]) in order to guarantee the availability and accessibility of the data despite host failures. As replication is highly storage demanding, recent approaches suggested the use of erasure-codes to ofer the same fault-tolerance while optimizing storage usage at the hosts. Initial works focused on a ix set of data hosts. To guarantee longevity and scalability, a storage service should be able to dynamically mask hosts failures by allowing new hosts to join, and failed host to be removed without service interruptions. This work presents the irst erasure-code based atomic algorithm, called Ares, which allows the set of hosts to be modiied in the course of an execution. Ares is composed of three main components: (i) a reconiguration protocol, (ii) a read/write protocol, and (iii) a set of data access primitives. The design of Ares is modular and is such to accommodate the usage of various erasure-code parameters on a per-coniguration basis. We provide bounds on the latency of read/write operations, and analyze the storage and communication costs of the Ares algorithm.

show abstract

A Performance Evaluation of Erasure Coding Libraries for Cloud-Based Data Stores

Cited by 12 publications

References 10 publications

Secure Secret Sharing With Adaptive Bandwidth in Distributed Cloud Storage Systems

Secure Secret Sharing With Adaptive Bandwidth in Distributed Cloud Storage Systems

Erasure coding for distributed storage systems

ARES: Adaptive, Reconfigurable, Erasure Coded, Atomic Storage

Contact Info

Product

Resources

About