Dynamic resource management on distributed systems using reconfigurable applications

Abstract

Efficient management of distributed resources, under conditions of unpredictable and varying workload, requires enforcement of dynamic resource management policies. Execution of such policies requires a relatively fine grain control over the resources allocated to jobs in the system. Although this is a difficult task using conventional job management and program execution models, reconfigurable applications can be used to make it viable. With reconfigurable applications, it is possible to dynamically change, during the course of program execution, the number of concurrently executing tasks of an application as well as the resources allocated. Thus, reconfigurable applications can adapt to internal changes in resource requirements and to external changes affecting available resources. In this paper, we discuss dynamic management of resources on distributed systems with the help of reconfigurable applications. We first characterize reconfigurable parallel applications. We then present a new programming model for reconfigurable applications and the Distributed Resource Management System (DRMS), an integrated environment for the design, development, execution, and resource scheduling of reconfigurable applications. Experiments were conducted to verify the functionality and performance of application reconfiguration under DRMS. A detailed breakdown of the costs in reconfiguration are presented with respect to several different applications. Our results indicate that application reconfiguration is effective under DRMS and can be beneficial in improving individual application performance as well as overall system performance. We observe a significant reduction in average job response time and an improvement in overall system utilization.This page intentionally left blank. AbstractEfficient management of distributed resources, under conditions of unpredictable and varying workload, requires enforcement of dynamic resource management policies. Execution of such policies requires a relatively fine grain control over the resources allocated to jobs in the system. Although this is a difficult task using conventional job management and program execution models, reconfigurable applications can be used to make it viable. With reconfigurable applications, it is possible to dynamically change, during the course of program execution, the number of concurrently executing tasks of an application as well as the resources allocated. Thus, reconfigurable applications can adapt to internal changes in resource requirements and to external changes affecting available resources. In this paper, we discuss dynamic management of resources on distributed systems with the help of reconfigurable applications. We first characterize reconfigurable parallel applications. We then present a new programming model for reconfigurable applications and the Distributed Resource Management System (DRMS), an integrated environment for the design, development, execution, and resource scheduling of reconfigurable applications. Experiments were conducted to ver...

A multi-layered resource management framework for dynamic resource management in enterprise DRE systems

“…For example, DRM capabilities have been applied to specific layers, such as the application layer [22] or network layer [9]. DRM capabilities have also been applied either in standalone [23] or distributed [24] deployments.…”

“…Application-driven distributed DRM was described in [22], where the authors leverage the reconfigurability of their parallel applications to dynamically change the number of executing tasks and allocate resources based on changes in resource availability. This work deals with systems in which the arrival of jobs is unpredictable and highly variable, which is similar to the shipboard computing scenarios that are the focus of this paper.…”

Dynamic Data Distribution and Processor Repartitioning for Irregularly Structured Computations

“…In this paper, we will primarily be concerned with the runtime mechanisms that support application reconfiguration. The other aspects of DRMS, including the resource management infrastructure, are described elsewhere [19,25]. An online overview of DRMS is available at the URL: http://www.research.ibm.com/ drms.…”

“…To make this promise of better performance through resource reconfiguration into a reality, we have designed and implemented the Distributed Resource Management System (DRMS) [19,25]. DRMS efficiently manages system resources under dynamically changing application demands.…”

Exploiting application tunability for efficient, predictable parallel resource management

“…The DRMS (Distributed Resource Management System) [14] is an exception, providing system support and resource management for dynamic reconfiguration of parallel programs (with respect to the number of tasks and mapping of tasks to processors). This reconfiguration can be thought of as being closely related to our notion of application tunability; however, unlike the focus of this paper, DRMS exploits this reconfigurability primarily to optimize system utilization and providing only best-effort guarantees to the application.…”