Efficient cooperative caching using hints

Sarkar, Prasenjit; Hartman, John H.

doi:10.1145/238721.238741

Cited by 91 publications

(55 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our proposed scientific data caching can be viewed as extended cooperative caching [14,18,20,44]: it pools secondary storage in a LAN environment to reduce access misses that require wide-area data transfers from external data sources. In addition, datasets in the cache are likely to be from different external data repositories.…”

Section: Related Workmentioning

confidence: 99%

Improving Data Availability for Better Access Performance: A Study on Caching Scientific Data on Distributed Desktop Workstations

Vazhkudai

Zhang

2009

J Grid Computing

View full text Add to dashboard Cite

Client-side data caching serves as an excellent mechanism to store and analyze the rapidly growing scientific data, motivating distributed, client-side caches built from unreliable desktop storage contributions to store and access large scientific data. They offer several desirable properties, such as performance impedance matching, improved space utilization, and high parallel I/O bandwidth. In this context, we are faced with two key challenges: (1) the finite amount of contributed cache space is stretched This work builds on several of our previously published studies. While we included extended introduction of the FreeLoader framework [57,58] as well as our combination of prefix caching and collective download techniques [33], this paper proposes the novel RDPR technique as well as a new striping-enabled cache management algorithm. We also discussed methods to combine RDPR with prefix caching and collective download techniques. Overall, more than 60% of the manuscript's content has not been previously published.X. Ma · Z. Zhang (B) North Carolina State University, Raleigh, NC, USA e-mail: zzhang3@ncsu.edu X. Ma · S. S. Vazhkudai Oak Ridge National Laboratory, Oak Ridge, TN, USA by the ever increasing scientific dataset sizes and (2) the transient nature of volunteered storage nodes impacts data availability. In this article, we address these challenges by exploiting the existence of external, primary copies of datasets. We propose a novel combination of prefix caching, collective download, and remote partial data recovery (RPDR), to deal with optimal cache space consumption and storage node volatility. Our evaluation, performed on our FreeLoader prototype, indicates that prefix caching can significantly improve the cache hit rate and partial data recovery is better than (or comparable to) many persistent-data availability techniques.

show abstract

Section: Related Workmentioning

confidence: 99%

Improving Data Availability for Better Access Performance: A Study on Caching Scientific Data on Distributed Desktop Workstations

Vazhkudai

Zhang

2009

J Grid Computing

View full text Add to dashboard Cite

show abstract

“…The cooperative cache [4,5,6] was proposed to reduce servers' load and to get high performance. In the cooperative cache, if a file system cache in a client doesn't handle a request to a file, the client sends the request to the other client's cache that caches the file rather than to the server because the access time of another client's memory is faster than that of the server's disk.…”

Section: Cooperative Cachementioning

confidence: 99%

“…Dahlin et al [4] suggested the efficient cache management scheme called N-chance algorithm, Feeley et al [5] suggested another efficient cache management scheme called modified Nchance algorithm in GMS (Global Memory Service). Sarkar et al [6] suggested the hint-based cooperative caching to reduce the management overhead using hint. Thus, the hint-based cooperative cache is scalable and can be adopted in the large-scale system such as cluster computer.…”

Section: Cooperative Cachementioning

confidence: 99%

“…Unlike previous hint-based cooperative cache research [6], we managed information and cached blocks per block, not per file. Because many clients share large files among them in a parallel file system, it is more adaptable to manage information and to cache per block than per file in Coopc-PVFS.…”

Section: Information Managementmentioning

confidence: 99%

“…Among researches for distributed file systems, the cooperative cache [4,5,6] was proposed to reduce servers' load and to get high performance. Because the access time of another client's memory is faster than that of a server's disk, to get the block from another client's memory has faster response time than to get a block from a server's disk.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Design and Implementation of the Cooperative Cache for PVFS

Hwang

Kim

Jung

et al. 2004

Computational Science - ICCS 2004

View full text Add to dashboard Cite

Abstract.Recently, there have been many efforts to get high performance in cluster computing with inexpensive PCs connected through high-speed networks. Some of them were to provide high bandwidth and parallelism in file service using a distributed file system. Other researches for distributed file systems include the cooperative cache that reduces servers' load and improves overall performance. The cooperative cache shares file caches among clients so that a client can request a file to another client, not to the server, through interclient message passing. In various distributed file systems, PVFS (Parallel Virtual File System) provides high performance with parallel I/O in Linux widely used in cluster computing. However, PVFS doesn't support any file cache facility. This paper describes the design and implementation of the cooperative cache for PVFS (Coopc-PVFS). We show the efficiency of Coopc-PVFS in comparison to original PVFS. As a result, the response time of Coopc-PVFS is shorter than or similar to that of original PVFS.

show abstract

Dynamo: design, implementation, and evaluation of cooperative persistent object management in a local area network

Yang

Wang

Nittel

et al. 2000

Softw: Pract. Exper.

View full text Add to dashboard Cite

In light of advances in processor and networking technology, especially the emergence of network attached disks, the traditional client-server architecture of persistent storage systems has become suboptimal for many computation/data intensive applications. In this paper, we introduce a revised architecture for persistent object management employing network attached storage: the dynamic object server environment (Dynamo). Dynamo introduces two main architectural innovations: (1) to provide high scalability, the object management functions are mainly performed cooperatively by the clients in the system. Furthermore, data is transferred directly to the client's cache from network-attached disks, thus avoiding copies from a disk to the server buffer and then over the network to the client. Other systems have an a priori assignment of 'object management' functions to specific nodes; (2) Dynamo uses a cooperative cache management which employs a decentralized lottery-based page replacement strategy with a novel technique which attempts to put a page replaced from one node into the cache of another node which is likely to access the page in the near future. We show via performance benchmarks run on the Dynamo system and simulation results how this architecture increases the system's adaptability, scalability and cost performance. LAN is a less urgent issue, we assume that whether a node should assume the responsibility for managing a set of data can be determined solely on the basis of performance (e.g. whether this node has enough computer resources), reliability, etc. 3. Symmetric access time. We assume that the LAN is connected via a high speed communication network, such as Ethernet, Myrinet, Fibre Channel Arbitrated Loops, etc. In addition, the average number of nodes in a LAN ranges from ten to several hundred, and the topologies of the communication network is often straightforward. As a result, the access time for node A to access the data in node B's memory is more or less the same as to access the data in node C's memory. Moreover, the access time is several orders of magnitude less than to access data from DYNAMO Coordinator and object management CoordinatorAs the name implies, the coordinator has a delegating, coordinating, and book keeping role in Dynamo. It manages the information that has to be kept centrally for all nodes; however, access to this information is kept minimal to avoid the analog of the server bottleneck. According to the size, the system can employ a single, central coordinator or a distributed coordinator. In the case of a distributed coordinator scheme, the set of nodes running a coordinator is known to all nodes in the LAN. The coordinator has several responsibilities, including: (1) keeping track of all Network-Attached Storage (NAS) space in the LAN; and (2) keeping track of which object manager manages which data.

show abstract

Efficient cooperative caching using hints

Cited by 91 publications

References 11 publications

Improving Data Availability for Better Access Performance: A Study on Caching Scientific Data on Distributed Desktop Workstations

Improving Data Availability for Better Access Performance: A Study on Caching Scientific Data on Distributed Desktop Workstations

Design and Implementation of the Cooperative Cache for PVFS

Dynamo: design, implementation, and evaluation of cooperative persistent object management in a local area network

Contact Info

Product

Resources

About