Wide-area cooperative storage with CFS

Dabek, Frank; Kaashoek, M. Frans; Karger, David R.; Morris, Robert; Stoica, Ion

doi:10.1145/502051.502054

Cited by 446 publications

(165 citation statements)

References 0 publications

Supporting

Mentioning

155

Contrasting

Unclassified

Order By: Relevance

“…Additionally peers may cache objects according to their hard drive space using the Greedy-Dual Size replacement policy. In CFS [28] and LessLog [29], a similar replication strategy is used; however, objects are split into blocks and replication is performed on a block basis. Other works consider physical information as well on selecting the replication peers.…”

Section: Related Workmentioning

confidence: 99%

SeekStream: adapting to dynamic user behavior in P2P video‐on‐demand

Deltouzos

Gkortsilas

Efthymiopoulos

et al. 2016

Int J Communication

View full text Add to dashboard Cite

In recent years, the users' perceived quality of experience (QoE) in streaming services has gained a lot of attention. Particularly, a number of research efforts have focused on providing live streaming and video-ondemand (VoD) services using peer-to-peer (P2P) architectures. However, in these proposed architectures, the heterogeneity of users and their dynamic behavior has not been sufficiently studied. In a real life scenario, where users have highly heterogeneous bandwidth resources (cable, DSL, 3G networks, etc) and can arbitrarily decide to perform a VCR function (stop, fast forward and seeking), ignoring this behavior can significantly deteriorate the system's efficiency and the perceived QoE. In this paper, we present SeekStream, a scalable P2P VoD architecture that ensures the stable delivery of the video stream to every participating user even in cases of high heterogeneity and frequent seeking operations. Specifically, SeekStream is a set of algorithms that optimize the P2P overlay dynamically and in a distributed fashion, making it adaptive to users dynamic behavior and bandwidth changes. The available bandwidth resources of the participating users are optimally exploited, keeping the contribution from the media server(s) to a minimal level. To illustrate the performance of the proposed algorithms, we are using a centralized overlay network manager that discovers the optimal network graph as a reference. We have developed an extensive P2P VoD simulator that shows the efficiency, scalability, and stability of our system under variant and dynamic conditions. The algorithms of our proposed system introduce less than 4% bandwidth overhead while we achieve high offloading of the media server(s). SeekStream guarantees a high block reception rate for the users, even under extreme seeking patterns.In P2P streaming services, each media object (video) is assumed to have a certain average playback rate and is usually divided into blocks of equal size. Peers that are interested to watch the video may enter the system at arbitrary time instants and consume the video fraction(s) of their interest.Adequate space is allocated in each peer's hard disk for the purpose of saving these blocks, which are provided either directly from the media server or from the participating peers. Generally, peers join the system and watch the video sequentially. However, it is not uncommon for someone to skip one or more parts of the video that are of no interest to it.For example, in Figure 1, we depict the hard disk of peer i watching a video of 64 blocks. Peer i joined the system and opted to watch the video not from its beginning, but starting from block 5. Then after having watched until block 12, it jumped to block 17, then from 28 to 37. Its current playback point is block 50 at time t 0 . Because of the jumps, we can see that the white blocks have not been requested, thus creating blank areas, whereas the blocks that are pending (requested but not yet delivered) and received are orange and green respectively. Different video par...

show abstract

Section: Related Workmentioning

confidence: 99%

SeekStream: adapting to dynamic user behavior in P2P video‐on‐demand

Deltouzos

Gkortsilas

Efthymiopoulos

et al. 2016

Int J Communication

View full text Add to dashboard Cite

show abstract

“…Many P2P file systems [12][13][14][15][16] use storage and routing based on DHT [10,39]. A block is hashed to an address that fully determines the locations of block's replicas.…”

Section: Related Workmentioning

confidence: 99%

“…The system can take advantage of the geographical dispersion of the resources, thus offering better protection in case of theft or natural disasters (e.g., fire or flood). Finally, P2P solutions have already proved to work well in enterprise environments (GFS [7], MapReduce [8], Astrolabe [9], DHT [10] used in HYDRAstor [11], etc. ).…”

Section: Introductionmentioning

confidence: 99%

“…In case of large amounts of data begin backed up, the throughput of the network might also become a limiting factor (especially when combined with low availability of the machines). On the other hand, our control and synchronization mechanisms are based on DHT and gossiping mechanisms, which scalability was often discussed and proved to be efficient (see [10,39,60,61] and references inside).…”

mentioning

confidence: 99%

See 1 more Smart Citation

Flexible replica placement for optimized P2P backup on heterogeneous, unreliable machines

Skowron

Rządca

2015

Concurrency and Computation

View full text Add to dashboard Cite

SUMMARYP2P architecture is a viable option for enterprise backup. In contrast to dedicated backup servers, nowadays, a standard solution, making backups directly on organization's workstations should be cheaper as existing hardware is used, more efficient as there is no single bottleneck server, and more reliable as the machines can be geographically dispersed. We present an architecture of a P2P backup system that uses pairwise replication contracts between a data owner and a replicator. In contrast to a standard P2P storage system using directly a distributed hash table (DHT), the contracts allow our system to optimize replicas' placement depending on a specific optimization strategy and so to take advantage of the heterogeneity of the machines and the network. Such optimization is particularly appealing in the context of backup: replicas can be geographically dispersed, the load sent over the network can be minimized, or the optimization goal can be to minimize the backup/restore time. However, managing the contracts, keeping them consistent and adjusting them in response to dynamically changing environment is challenging. We built a scientific prototype and ran experiments on 150 workstations in our university's computer laboratories and, separately, on 50 PlanetLab nodes. We found out that the main factor affecting the performance of the system is the availability of the machines. Yet, our main conclusion is that it is possible to build an efficient and reliable backup system on highly unavailable machines, as our computers had just 13% average availability.

show abstract

“…As a peer-to-peer storage system, CFS [15] stores data blocks reliably using distributed hash tables (DHash). DHash arranges blocks over a large number of servers to provide high scalability, fault tolerance and load balancing.…”

Section: Related Workmentioning

confidence: 99%

Analyzing the feasibility of building a new mass storage system on distributed resources

Huang

Karpovich

Grimshaw

2007

Concurrency and Computation

View full text Add to dashboard Cite

SUMMARYThe average PC now contains a large and increasing amount of storage with an ever greater amount left unused. We believe there is an opportunity for organizations to harness the vast unused storage capacity on their PCs to create a very large, low cost, shared storage system. What is needed is the proper storage system architecture and software to exploit and manage the unused portions of existing PC storage devices across an organization and make it reliably accessible to users and applications. We call our vision of such a storage system Storage@desk (SD). This paper describes our first step towards the realization of Storage@desk -a study of machine and storage characteristics and usage in a model organization. We studied 729 PCs in an academic institution for 91 days, monitoring the configuration, load and usage of the major machine subsystems, i.e. disk, memory, CPU and network. To further analyze the availability characteristics of storage in an SD system, we performed a trace-driven simulation of some basic storage allocation strategies. This paper presents the results of our data collection efforts, our analysis of the data, our simulation results, and our conclusion that a Storage@desk system is indeed feasible and holds promise as a cost effective way to create massive storage systems.

show abstract

Wide-area cooperative storage with CFS

Cited by 446 publications

References 0 publications

SeekStream: adapting to dynamic user behavior in P2P video‐on‐demand

SeekStream: adapting to dynamic user behavior in P2P video‐on‐demand

Flexible replica placement for optimized P2P backup on heterogeneous, unreliable machines

Analyzing the feasibility of building a new mass storage system on distributed resources

Contact Info

Product

Resources

About