A Survey and Classification of Software-Defined Storage Systems

Macedo, Ricardo; Paulo, João; Pereira, José; Bessani, Alysson

doi:10.1145/3385896

Cited by 21 publications

(13 citation statements)

References 84 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…SDS uses software abstractions to provide provisioning and data management indenpent of the underyling hardware. In these fields we see notable surveys that describe a similar problem of data increase and performance requirements [55,80].…”

Section: Related Workmentioning

confidence: 99%

Past, Present and Future of Computational Storage: A Survey

Lukken¹,

Trivedi²

2021

Preprint

View full text Add to dashboard Cite

We live in a data-centric world where we are heading to generate close to 200 Zettabytes of data by the year 2025 [56]. Our data processing requirements have also increased as we push to build data processing frameworks that can process large volumes of data in a short duration, a few milli-and even micro-seconds. In the prevalent computer systems designs, data is stored passively in storage devices which is brought in for processing and then the results are written out. As the volume of data explodes this constant data movement has led to a data movement wall which hinders further process and optimizations in data processing systems designs. One promising alternative to this architecture is to push computation to the data (instead of the other way around), and design a computational-storage device or CSD. The idea of CSD is not new and can trace its root to the pioneering work done in the 1970s and 1990s. More recently, with the emergence of non-volatile memory (NVM) storage in the mainstream computing (e.g., NAND flash and Optane), the idea has again gained a lot of traction with multiple academic and commercial prototypes being available now. In this brief survey we present a systematic analysis of work done in the area of computation storage and present future directions.

show abstract

Section: Related Workmentioning

confidence: 99%

Past, Present and Future of Computational Storage: A Survey

Lukken¹,

Trivedi²

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Through this interface, the control plane is able to define how PAIO stages handle I/O requests. Nonetheless, concerns related to the coordination and dependability of data plane stages, as well as the resolution of conflicting policies and layers are responsibility of the control plane [21], and are thus orthogonal to this paper.…”

Section: Interfacesmentioning

confidence: 99%

“…The Software-Defined Storage (SDS) [21,32] paradigm promises an appealing solution to these limitations. It aims at decoupling I/O functionality into two planes: control and data.…”

Section: Introductionmentioning

confidence: 99%

“…First, I/O optimizations are tightly integrated within the core of each solution, making it challenging to port these to other systems with similar performance goals. Second, in shared environments where multiple systems operate concurrently and compete for shared resources, individual optimizations can conflict with each other [15], leading to I/O contention and performance variation [29,35].The Software-Defined Storage (SDS) [21,32] paradigm promises an appealing solution to these limitations. It aims at decoupling I/O functionality into two planes: control and data.…”

mentioning

confidence: 99%

See 1 more Smart Citation

PAIO: A Software-Defined Storage Data Plane Framework

Macedo¹,

Tanimura²,

Haga³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

We propose PAIO, the first general-purpose framework that enables system designers to build custom-made Software-Defined Storage (SDS) data plane stages. It provides the means to implement storage optimizations adaptable to different workflows and user-defined policies, and allows straightforward integration with existing applications and I/O layers. PAIO allows stages to be integrated with modern SDS control planes to ensure holistic control and system-wide optimal performance. We demonstrate the performance and applicability of PAIO with two use cases. The first improves 99 th percentile latency by 4× in industry-standard LSM-based key-value stores. The second ensures dynamic per-application bandwidth guarantees under shared storage environments. IntroductionData-centric systems such as databases, key-value stores (KVS), and machine learning engines, share the need for efficient data storage and retrieval. This has led to the implementation of isolated I/O optimizations (e.g., scheduling, differentiation, caching) to address their storage requirements, such as resource fairness and throughput/latency SLOs [9,28,36]. This approach, however, has two main drawbacks. First, I/O optimizations are tightly integrated within the core of each solution, making it challenging to port these to other systems with similar performance goals. Second, in shared environments where multiple systems operate concurrently and compete for shared resources, individual optimizations can conflict with each other [15], leading to I/O contention and performance variation [29,35].The Software-Defined Storage (SDS) [21,32] paradigm promises an appealing solution to these limitations. It aims at decoupling I/O functionality into two planes: control and data. The control plane is a logically centralized entity with

show abstract

“…First, disaggregated architectures in the cloud and datacenters are increasingly offering some form of in-storage processing [33,8] and flexible data management [17,13] with high-speed network connectivity. Second, as an effort to keep the management, configuration and monitoring of a large number of storage nodes scalable, Software-Defined Storage (SDS) has been proposed [39] -albeit the goal of existing SDS systems is almost exclusively to guarantee performance isolation and service levels in distributed multi-tenant settings [27].…”

Section: Introductionmentioning

confidence: 99%

Towards Software-Defined Data Protection: GDPR Compliance at the Storage Layer is Within Reach

István¹,

Ponnapalli²,

Chidambaram³

2020

Preprint

View full text Add to dashboard Cite

Enforcing data protection and privacy rules within large data processing applications is becoming increasingly important, especially in the light of GDPR and similar regulatory frameworks. Most modern data processing happens on top of a distributed storage layer, and securing this layer against accidental or malicious misuse is crucial to ensuring global privacy guarantees. However, the performance overhead and the additional complexity for this is often assumed to be significant -in this work we describe a path forward that tackles both challenges. We propose "Software-Defined Data Protection" (SDP), an adoption of the "Software-Defined Storage" approach to non-performance aspects: a trusted controller translates company and application-specific policies to a set of rules deployed on the storage nodes. These, in turn, apply the rules at line-rate but do not take any decisions on their own. Such an approach decouples often changing policies from request-level enforcement and allows storage nodes to implement the latter more efficiently.Even though in-storage processing brings challenges, mainly because it can jeopardize line-rate processing, we argue that today's Smart Storage solutions can already implement the required functionality, thanks to the separation of concerns introduced by SDP. We highlight the challenges that remain, especially that of trusting the storage nodes. These need to be tackled before we can reach widespread adoption in cloud environments.

show abstract

A Survey and Classification of Software-Defined Storage Systems

Cited by 21 publications

References 84 publications

Past, Present and Future of Computational Storage: A Survey

Past, Present and Future of Computational Storage: A Survey

PAIO: A Software-Defined Storage Data Plane Framework

Towards Software-Defined Data Protection: GDPR Compliance at the Storage Layer is Within Reach

Contact Info

Product

Resources

About