HARNESS: Heterogeneous Adaptable Reconfigurable NEtworked SystemS

Dongarra, Jack; Fagg, Graham E.; Geist, Alessandro; Kohl, James Arthur; Papadopoulos, Philip M.; Scott, Sarah; Sunderam, Vaidy S.; Magliardi, M.

doi:10.1109/hpdc.1998.710029

Cited by 16 publications

(10 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…SSI modules for the underlying networks can be activated at runtime to deal with network heterogeneity across clusters. HARNESS [2] provides mechanisms for applications to collaborate with each other in heterogeneous environment through a pluggable model that allows multiple distributed virtual machines to merge, split and interact with each other. A commercial product, which is MPI Connect, from Scali [10] supports multiple underlying networks at runtime.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Design and Implementation of an RDMA Gateway for Heterogeneous Clusters

Kim¹,

Han²,

Jung³

et al. 2007

2007 International Conference on Convergence Information Technology (ICCIT 2007)

View full text Add to dashboard Cite

Building high-performance clusters using one of the two leading network technologies, Myrinet and Infini-Band, has been thought as a de facto way to achieve several teraflops computing power. Meanwhile, maintaining both types of clusters, it appears, may have created an another challenge for the MPI programming system, the most popular parallel programming library that has been successfully used on both networks. The belief that extending cluster resources across two different types of networks may increase computing parallelism has driven many researchers to tackle this challenge with various viewpoints. We approach this challenge with a different perspective, application transparency, which is accomplishing the goal without any modification of legacy MPI applications. We, therefore, focus on the design of an RDMA gateway that can relay messages very fast, and this design focus turns out to be a better way to preserve the application transparency. RDMA Gateway (RG), our prototyped system, has a very efficient memory management mechanism that prevents RG from showing irregular spikes of a memory usage under a heavy load condition. Experimental results show that running parallel applications over heterogeneous clusters can be very promising with low performance overhead.

show abstract

Section: Related Workmentioning

confidence: 99%

“…M emorySpace eager = N connection * 16 * 16(KB) (2) However, the way to use the circular buffer does not guarantee that our system is scalable. For instance, if there are 128 processes in each cluster, the amount of required memory is about 4 GB (2*128*128*16*16 KB).…”

Section: Protocolsmentioning

confidence: 99%

Design and Implementation of an RDMA Gateway for Heterogeneous Clusters

Kim¹,

Han²,

Jung³

et al. 2007

2007 International Conference on Convergence Information Technology (ICCIT 2007)

View full text Add to dashboard Cite

show abstract

“…Yet the resource demands of simulation applications require a growing variety of heterogeneous clusters and computational grids. Accordingly, next-generation scientific software must stretch its current bounds and become more dynamic, interactive, interoperable, fault-tolerant and mobile.Harness [1,2] is an adaptable environment for next-generation heterogeneous distributed computing. Using a fundamentally new "pluggable" heterogeneous virtual machine model, as a followon to PVM [3], Harness provides a flexible platform for developing dynamic, fault-tolerant, parallel software applications.…”

mentioning

confidence: 99%

Harness: Adaptable Virtual Machine Environment for Heterogeneous Cluster

Geist

Kohl

Scott

et al. 1999

Parallel Process. Lett.

View full text Add to dashboard Cite

This paper describes ongoing work on the Harness system for next-generation heterogeneous distributed computing. Harness is an adaptable, reliable virtual machine environment being built as a follow-on to PVM. The three fundamental concepts presented here are parallel plugins, fault-tolerant distributed control, and dynamically merging and splitting virtual machines. The distributed control mechanisms provide the support framework necessary for coordinating and applying parallel plugins that allow applications to customize or tune their operating environment on-the-fly. In the spirit of CUMULVS, Harness applications can plug into each other to couple for collaborative computing. Virtual machines that merge and split can assist applications in dynamically utilizing different computing resources to suit changing computational needs. IntroductionSimulation science is a field of increasing importance, as more flexible, efficient and cost-effective software solutions begin to replace traditional physical experiments and prototypes. Online simulation experiments provide a forum for collaboration among scientific teams across wide-area networks. Yet the resource demands of simulation applications require a growing variety of heterogeneous clusters and computational grids. Accordingly, next-generation scientific software must stretch its current bounds and become more dynamic, interactive, interoperable, fault-tolerant and mobile.Harness [1,2] is an adaptable environment for next-generation heterogeneous distributed computing. Using a fundamentally new "pluggable" heterogeneous virtual machine model, as a followon to PVM [3], Harness provides a flexible platform for developing dynamic, fault-tolerant, parallel software applications. Harness pluggability extends the common serial plugin concept and applies it to the parallel computing case, allowing services, applications and resources to be plugged together as needed by state-of-the-art simulations. Applications can customize or tune their operating environments on-the-fly to cater to their potentially changing computational needs. Coupled applications can be assembled by plugging together different component simulation models, as is being

show abstract

“…It could be a huge loss of resources if private clusters are excluded from Grid due to the reason above. Hence there has been much work on making parallel programs be executed over clusters in private networks [1,2,3]. Most of them relay messages using a separate application because it could be a more general solution than kernel-level strategy.…”

Section: Introductionmentioning

confidence: 99%

Universal Execution of Parallel Processes: Penetrating NATs over the Grid

Han

Yeom

et al. 2007

Computational Science – ICCS 2007

View full text Add to dashboard Cite

Abstract. Today, clusters are very important computing resources and many computing centers manage their clusters in private networks. But parallel programs may not work in private clusters. Because hosts in private clusters are not globally reachable, hosts behind different private clusters cannot be reached directly in order to communicate. It will certainly be a huge loss of resources if private clusters are excluded from the computing due to this reason. There has been much research on this issue, but most of them concentrate on user-level relaying because it is a general and easily-implementable solution. However, even wellimplemented, user-level solutions result in much longer communication latency than kernel-level solutions. This paper adopted a novel kernellevel solution and applied it to MPICH-G2. Our scheme is generally applicable, simple and efficient. The experimental results show that our scheme incurs very little overhead except when small messages are transmitted. That is, it supports a more universal computing environment by including private clusters with remarkably little overhead.

show abstract

HARNESS: Heterogeneous Adaptable Reconfigurable NEtworked SystemS

Cited by 16 publications

References 0 publications

Design and Implementation of an RDMA Gateway for Heterogeneous Clusters

Design and Implementation of an RDMA Gateway for Heterogeneous Clusters

Harness: Adaptable Virtual Machine Environment for Heterogeneous Cluster

Universal Execution of Parallel Processes: Penetrating NATs over the Grid

Contact Info

Product

Resources

About