Utopia: A load sharing facility for large, heterogeneous distributed computer systems

Abstract

Load sharing in large, heterogeneous distributed systems allows users to access vast amounts of computing resources scattered around the system and may provide substantial performance improvements to applications. We discuss the design and implementation issues in Utopia, a load sharing facility specifically built for large and heterogeneous systems. The system has no restriction on the types of tasks that can be remotely executed, involves few application changes and no operating system change, supports a high degree of transparency for remote task execution, and incurs low overhead. The algorithms for managing resource load information and task placement take advantage of the clustering nature of large‐scale distributed systems; centralized algorithms are used within host clusters, and directed graph algorithms are used among the clusters to make Utopia scalable to thousands of hosts. Task placements in Utopia exploit the heterogeneous hosts and consider varying resource demands of the tasks. A range of mechanisms for remote execution is available in Utopia that provides varying degrees of transparency and efficiency. A number of applications have been developed for Utopia, ranging from a load sharing command interpreter, to parallel and distributed applications, to a distributed batch facility. For example, an enhanced Unix command interpreter allows arbitrary commands and user jobs to be executed remotely, and a parallel make facility achieves speed‐ups of 15 or more by processing a collection of tasks in parallel on a number of hosts.

InstantGrid: A Framework for On-Demand Grid Point Construction

“…Job schedulers (e.g., [3], [20], [21], etc.) have been a widely adopted approach to aggregate and share distributed resources.…”

Efficient use of parallel libraries on heterogeneous Networks of Workstations

“…However, their aim is to reach the net maximum e ciency, rather than o ering a speci®c support for parallel programming, which is possible, but not privileged. It is worth mentioning UTOPIA [12] as well, a load sharing facility implemented on top of Unix for very large and heterogeneous systems. As well as LSF it is a general purpose user transparent system, which supports remote execution only at job initiation time.…”

“…As discussed in [12], three kinds of heterogeneity can be considered, that is: · Con®guration: hosts have the same architecture but di erent con®guration, · Architectural: executable ®les cannot be exchanged, and · Operating System heterogeneity.…”

Speeding up simulation applications using WinGrid

“…Desktop grid applications such as Platform LSF (Zhou, 1992) The practice of CSP-based simulation can widely benefit from Grid computing (Taylor et al, 2004;Mustafee, 2007). By means of two case studies, done in conjunction with a major automobile manufacturing company (the Ford Motor Company) and a leading European investment bank respectively, we investigate how a desktop grid implemented with our system WinGrid can increase the performance of both discrete event and Monte Carlo simulation experimentation.…”

“…Internet computing seeks to provide resource virtualization through aggregation of idle CPU cycles of the PCs connected over the Internet and the Local Area Network (LAN). When this form of computing is confined to an enterprise and the purpose of resource virtualization is to support the execution of enterprises' applications then we use the term enterprise desktop grids (Chien et al, 2003).Desktop grid applications such as Platform LSF (Zhou, 1992) The practice of CSP-based simulation can widely benefit from Grid computing (Taylor et al, 2004;Mustafee, 2007). By means of two case studies, done in conjunction with a major automobile manufacturing company (the Ford Motor Company) and a leading European investment bank respectively, we investigate how a desktop grid implemented with our system WinGrid can increase the performance of both discrete event and Monte Carlo simulation experimentation.…”