The Kangaroo approach to data movement on the Grid

Thain, Douglas; Basney, Jim; Son, Sechang; Livny, Miron

doi:10.1109/hpdc.2001.945200

Cited by 79 publications

(62 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In such cases, checkpoint data or results from computations thus far may have to be moved to the source in an optimal fashion from many locations. Kangaroo, a hop-based checkpoint management system, from Condor addresses some issues therein [TBS+01]. Thus, in order to facilitate efficient bulk data movement, we need to address several of the aforementioned system and transfer challenges.…”

Section: Conclusion and Discussionmentioning

confidence: 99%

Orchestrating Bulk Data Movement in Grid Environments

Vazhkudai

2005

View full text Add to dashboard Cite

Section: Conclusion and Discussionmentioning

confidence: 99%

Orchestrating Bulk Data Movement in Grid Environments

Vazhkudai

2005

View full text Add to dashboard Cite

“…Though Condor-G by itself does not provide any data access functions, it can interface to services such as Kangaroo [47] and Stork [48] that enable it to mediate access to remote files and manage data transfers. Condor-G allows for creation of applications belonging to different models such as workflows and supports different scheduling strategies.…”

Section: Condor-gmentioning

confidence: 99%

Designing a resource broker for heterogeneous grids

Venugopal¹,

Nadiminti²,

Gibbins³

et al. 2007

Softw Pract Exp

View full text Add to dashboard Cite

Abstract:Grids provide uniform access to aggregations of heterogeneous resources and services such as computers, networks and storage owned by multiple organizations. However, such a dynamic environment poses many challenges for application composition and deployment. In this paper, we present the design of the Gridbus Grid resource broker that allows users to create applications and specify different objectives through different interfaces without having to deal with the complexity of Grid infrastructure. We present the unique requirements that motivated our design and discuss how these provide flexibility in extending the functionality of the broker to support different low-level middlewares and user interfaces. We evaluate the broker with different job profiles and Grid middleware and conclude with the lessons learnt from our development experience.

show abstract

“…Various techniques have been proposed, apart from the ones mentioned above, centralized or in the context of the peer-to-peer paradigm. Kangaroo [24] is a data transfer system that aims at better overall performance by making opportunistic use of a chain of servers. The Composite Endpoint Protocol [26] collects high-level transfer data provided by the user and generates a schedule which optimizes the transfer performance by producing a balanced weighting of a directed graph.…”

Section: Transferring Filesmentioning

confidence: 99%

Optimizing Data Management in Grid Environments

Zissimos

Doka

Chazapis

et al. 2009

On the Move to Meaningful Internet Systems: OTM 2009

View full text Add to dashboard Cite

Abstract. Grids currently serve as platforms for numerous scientific as well as business applications that generate and access vast amounts of data. In this paper, we address the need for efficient, scalable and robust data management in Grid environments. We propose a fully decentralized and adaptive mechanism comprising of two components: A Distributed Replica Location Service (DRLS ) and a data transfer mechanism called GridTorrent. They both adopt Peer-to-Peer techniques in order to overcome performance bottlenecks and single points of failure. On one hand, DRLS ensures resilience by relying on a Byzantine-tolerant protocol and is able to handle massive concurrent requests even during node churn. On the other hand, GridTorrent allows for maximum bandwidth utilization through collaborative sharing among the various data providers and consumers. The proposed integrated architecture is completely backwards-compatible with already deployed Grids. To demonstrate these points, experiments have been conducted in LAN as well as WAN environments under various workloads. The evaluation shows that our scheme vastly outperforms the conventional mechanisms in both efficiency (up to 10 times faster) and robustness in case of failures and flash crowd instances.

show abstract

The Kangaroo approach to data movement on the Grid

Abstract: Abstract

Cited by 79 publications

References 23 publications

Orchestrating Bulk Data Movement in Grid Environments

Orchestrating Bulk Data Movement in Grid Environments

Designing a resource broker for heterogeneous grids

Optimizing Data Management in Grid Environments

Contact Info

Product

Resources

About