Resource Allocation for Steerable Parallel Parameter Searches

Faerman, M.; Birnbaum, Adam; Casanova, Henri; Berman, Francine

doi:10.1007/3-540-36133-2_14

Cited by 14 publications

(7 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These systems typically assume that the underlying simulations are independent and hence can be run in batch mode, ignoring interactivity considerations and without exploiting possibilities for reuse explored in this paper. Two exceptions are the work on Nimrod/O [1], which reuses previous results in a gradient descent-like guided search approach to identify the set of parameter points optimizing an objective function, and Virtual Instruments [8], which permit users to explicitly guide search of the parameter space by identifying regions of interest. Our SimX work combines elements of both systems but considerably extends them to enable reuse across user interactions, support more sophisticated objectives such as identification of Pareto-optimal points, and exploit reuse opportunities at the level of individual parameter point evaluations.…”

Section: Related Work and Discussionmentioning

confidence: 99%

Result reuse in design space exploration: A study in system support for interactive parallel computing

Yau

Damevski

Karamcheti

et al. 2008

2008 IEEE International Symposium on Parallel and Distributed Processing

View full text Add to dashboard Cite

This paper presents a system supporting reuse of simulation results in multi-experiment computational studies involving independent simulations and explores the benefits of such reuse. Using a SCIRun-based defibrillator device simulation code (DefibSim) and the SimX system for computational studies, this paper demonstrates how aggressive reuse between and within computational studies can enable interactive rates for such studies on a moderate-sized 128-node processor cluster; a brute-force approach to the problem would require two thousand nodes or more on a massively parallel machine for similar performance. Key to realizing these performance improvements is exploiting optimization opportunities that present themselves at the level of the overall workflow of the study as opposed to focusing on individual simulations. Such global optimization approaches are likely to become increasingly important with the shift towards interactive and universal parallel computing. BackgroundThe growing availability of low-cost, universal parallel computing resources, including commodity processors with 4-16 cores, GPUs with support for general-purpose computing and heterogeneous multicore chips, enables small groups of researchers or even individual researchers to have dedicated access to large amounts of compute power. This availability is expected to change usage models for parallel computing, shifting the traditional emphasis on batch calculations to interactive applications. Such change requires reconsidering how parallel software systems are structured * Supported by a collaborative NSF grant comprising CSR-0615225, CSR-0614770, and CSR-0615194. and how resources have to be managed for new interactive workflows.Our ongoing work with the SimX computational study system [27] provides a representative platform in which to investigate these issues. Recognizing that computer simulation has become an integral part of the scientific method, SimX supports a scientific exploration process that manifests itself as computational studies built out of multiple computational experiments corresponding to individual runs of simulation software. Examples of such studies range from exploration of design spaces in engineering to molecular simulations for drug design.As an example of the different considerations that come into play, the performance criterion driving the design of the SimX system is the need to provide meaningful results to the researcher at a timescale that permits the researcher to interactively drive the exploration process in studies involving tens of thousands of experiments. Satisfying this requirement requires managing resources at the level of the overall computational study instead of individual experiments. Unfortunately, with the few exceptions listed in Section 5, prior research has not examined how one might improve the performance of entire studies under high levels of parallelism. This paper describes SimX support for studylevel resource management using the specific context of Design Space Exploration (D...

show abstract

Section: Related Work and Discussionmentioning

confidence: 99%

Result reuse in design space exploration: A study in system support for interactive parallel computing

Yau

Damevski

Karamcheti

et al. 2008

2008 IEEE International Symposium on Parallel and Distributed Processing

View full text Add to dashboard Cite

show abstract

“…Note that these experiments did not include any user steering as our goal here is solely to verify and evaluate the VI software design. The software supports steering and we refer the reader to Faerman et al (2002) for a discussion of steering and scheduling.…”

Section: Methodsmentioning

confidence: 99%

The Virtual Instrument: Support for Grid-Enabled Mcell Simulations

Casanova¹,

Berman

Bartol³

et al. 2004

The International Journal of High Performance Computing Applica

Self Cite

View full text Add to dashboard Cite

Ensembles of widely distributed, heterogeneous resources, or Grids, have emerged as popular platforms for largescale scientific applications. In this paper we present the Virtual Instrument project, which provides an integrated application execution environment that enables end-users to run and interact with running scientific simulations on Grids. This work is performed in the specific context of MCell, a computational biology application. While MCell provides the basis for running simulations, its capabilities are currently limited in terms of scale, ease-of-use, and interactivity. These limitations preclude usage scenarios that are critical for scientific advances. Our goal is to create a scientific "Virtual Instrument" from MCell by allowing its users to transparently access Grid resources while being able to steer running simulations. In this paper, we motivate the Virtual Instrument project and discuss a number of relevant issues and accomplishments in the area of Grid software development and application scheduling. We then describe our software design and report on the current implementation. We verify and evaluate our design via experiments with MCell on a real-world Grid testbed.

show abstract

“…Scheduling of parameter sweep applications. Several grid computing infrastructures (APST [6], Nimrod [2,1], Condor [22], Globus [19,3], Netsolve [5], Virtual Instruments [10]) provide support for the scheduling of parameter sweep applications, where the same application is run with a change in parameter values across distributed resources. The underlying tools simplify the running of multiple simulation experiments, possibly running into the tens of thou- sands.…”

Section: Related Workmentioning

confidence: 99%

“…Sim-X generalizes this support to support more interactive exploration of fuzzier and dynamically changing optimization criteria. Closer to our goals is the scheduling module of the Virtual Instruments project [10], which implements priority-based resource allocation for the application's tasks. Sim-X extends this support to factor in reuse of results from previous tasks and control of the internal behavior of tasks.…”

Section: Related Workmentioning

confidence: 99%

“…Sim-X builds on several research efforts over the past decade on computational steering tools [20,12,11,23,4], domainspecific problem solving environments [14], and parametersweep tools introduced in the context of grid computing [6,2,1,22,19,3,5,10] to enable interactive computational studies. Sim-X uses the following key principles: Dynamic resource allocation.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Sim-X: parallel system software for interactive multi-experiment computational studies

Yau

Grinspun

Karamcheti

et al. 2006

Proceedings 20th IEEE International Parallel &Amp; Distributed Processing Symposium

View full text Add to dashboard Cite

Advances in high-performance computing have led to the broad use of computational studies in everyday engineering and scientific applications. A single study may require thousands of computational experiments, each corresponding to individual runs of simulation software with different parameter settings; in complex studies, the pattern of parameter changes is complex and may have to be adjusted by the user based on partial simulation results. Unfortunately, existing tools have limited high-level support for managing large ensembles of simultaneous computational experiments.In this paper, we present a system architecture for interactive computational studies targeting two goals. The first is to provide a framework for high-level user interaction with computational studies, rather than individual experiments; the second is to maximize the size of the studies that can be performed at close to interactive rates.We describe a prototype implementation of the system and demonstrate performance improvements obtained using our approach for a simple model problem.

show abstract

Resource Allocation for Steerable Parallel Parameter Searches

Cited by 14 publications

References 10 publications

Result reuse in design space exploration: A study in system support for interactive parallel computing

Result reuse in design space exploration: A study in system support for interactive parallel computing

The Virtual Instrument: Support for Grid-Enabled Mcell Simulations

Sim-X: parallel system software for interactive multi-experiment computational studies

Contact Info

Product

Resources

About