Grid resource management---CRUSH

Weil, Sage A.; Brandt, Scott; Miller, Ethan L.; Maltzahn, Carlos

doi:10.1145/1188455.1188582

Cited by 134 publications

(19 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Within a pool, the objects are sharded among aggregation units called placement groups (PGs). Depending on the replication factor, PGs are mapped to multiple OSDs using CRUSH, a pseudo-random data distribution algorithm [104]. Clients also use CRUSH to determine the OSD that should contain a given object, obviating the need for a centralized metadata service.…”

Section: Rados Scales To Thousands Of Object Storagementioning

confidence: 99%

The Case for Custom Storage Backends in Distributed Storage Systems

et al. 2020

View full text Add to dashboard Cite

For a decade, the Ceph distributed file system followed the conventional wisdom of building its storage backend on top of local file systems. This is a preferred choice for most distributed file systems today, because it allows them to benefit from the convenience and maturity of battle-tested code. Ceph's experience, however, shows that this comes at a high price. First, developing a zero-overhead transaction mechanism is challenging. Second, metadata performance at the local level can significantly affect performance at the distributed level. Third, supporting emerging storage hardware is painstakingly slow. Ceph addressed these issues with BlueStore, a new backend designed to run directly on raw storage devices. In only two years since its inception, BlueStore outperformed previous established backends and is adopted by 70% of users in production. By running in user space and fully controlling the I/O stack, it has enabled spaceefficient metadata and data checksums, fast overwrites of erasure-coded data, inline compression, decreased performance variability, and avoided a series of performance pitfalls of local file systems. Finally, it makes the adoption of backward-incompatible storage hardware possible, an important trait in a changing storage landscape that is learning to embrace hardware diversity.

show abstract

Section: Rados Scales To Thousands Of Object Storagementioning

confidence: 99%

The Case for Custom Storage Backends in Distributed Storage Systems

et al. 2020

View full text Add to dashboard Cite

show abstract

“…Within a pool, the objects are sharded among aggregation units called placement groups (PGs). Depending on the replication factor, PGs are mapped to multiple OSDs using CRUSH, a pseudo-random data distribution algorithm [99]. Clients also use CRUSH to determine the OSD that should contain a given object, obviating the need for a centralized metadata service.…”

Section: Objectsmentioning

confidence: 99%

File systems unfit as distributed storage backends

Aghayev

Weil

Kuchnik

et al. 2019

Proceedings of the 27th ACM Symposium on Operating Systems Principles

View full text Add to dashboard Cite

For a decade, the Ceph distributed file system followed the conventional wisdom of building its storage backend on top of local file systems. This is a preferred choice for most distributed file systems today because it allows them to benefit from the convenience and maturity of battle-tested code. Ceph's experience, however, shows that this comes at a high price. First, developing a zero-overhead transaction mechanism is challenging. Second, metadata performance at the local level can significantly affect performance at the distributed level. Third, supporting emerging storage hardware is painstakingly slow. Ceph addressed these issues with BlueStore, a new backend designed to run directly on raw storage devices. In only two years since its inception, BlueStore outperformed previous established backends and is adopted by 70% of users in production. By running in user space and fully controlling the I/O stack, it has enabled space-efficient metadata and data checksums, fast overwrites of erasure-coded data, inline compression, decreased performance variability, and avoided a series of performance pitfalls of local file systems. Finally, it makes the adoption of backwards-incompatible storage hardware possible, an important trait in a changing storage landscape that is learning to embrace hardware diversity. CCS Concepts • Information systems → Distributed storage; • Software and its engineering → File systems management; Software performance.

show abstract

“…Second, the object gateway poses a network bottleneck for clients writing to the Ceph cluster. One of the principal advantages of Ceph is that clients can write directly to OSDs by calculating where to store and retrieve data deterministically, via the CRUSH algorithm [20] and the cluster map, without being burdened by the network bottleneck of first connecting to a metadata server. By interposing itself between clients and OSDs, the Ceph Object Gateway re-introduces a network bottleneck, inhibiting performance.…”

Section: Designmentioning

confidence: 99%

Exploring Untrusted Distributed Storage for High Performance Computing

Smith

Riley

Syed

et al. 2019

Proceedings of the Practice and Experience in Advanced Research Computing on Rise of the Machines (Learning)

View full text Add to dashboard Cite

High performance computing systems are typically built with highthroughput and infrastructural uniformity in mind, but generally do not easily accommodate diverse data security requirements on a single cluster. Rather than fracturing that infrastructure by building many network isolated storage "islands" to secure each dataset covered by an individual data use agreement, we explore using the Ceph distributed storage system with client-side encryption to provision secure storage from a single, untrusted data lake.

show abstract

Grid resource management---CRUSH

Cited by 134 publications

References 16 publications

The Case for Custom Storage Backends in Distributed Storage Systems

The Case for Custom Storage Backends in Distributed Storage Systems

File systems unfit as distributed storage backends

Exploring Untrusted Distributed Storage for High Performance Computing

Contact Info

Product

Resources

About