Scalability Analysis of Job Scheduling Using Virtual Nodes

Bobroff, Norman; Coppinger, Richard; Fong, Liana; Seelam, Seetharami; Xu, Jing

doi:10.1007/978-3-642-04633-9_11

Cited by 6 publications

(3 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We use information about static workload data from the parallel workload archive (www.cs.huji.ac.il/labs/parallel/workload/) and from experiments reported in several publications (Krampe et al , 2010; Ernemann et al , 2003; Liu and Dong et al , 2010; Zhou, 2009). Moreover, these workload traces were used for the evaluation of different scheduling strategies for parallel systems (Krallmann et al , 1999; Bobroff et al , 2009; Schwiegelshohn and Yahyapour, 1998; Frachtenberg and Schwiegelshohn, 2008) and for grid research (Li, 2009; Mohammad Khanli et al , 2010; Sumathi and Gopalan, 2008; Li, 2005; Pan et al , 2005). These workload traces consist of information about all job submissions on a node for a certain period of time, which usually ranges over several months and several thousands of jobs.…”

Section: Literature Reviewmentioning

confidence: 99%

“…The grid scheduler must interact with the local schedulers managing computational resources and must adapt its behavior to the changing resources loads. Grid scheduling involves a series of challenging tasks (Bobroff et al , 2009). These include: searching for resources in the collection of geographically distributed nodes; and making scheduling decisions according to the required quality of service.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Colony of cooperating agents based independent job scheduling in a computation grid

Helmy

Rasheed

2011

International Journal of Intelligent Computing and Cybernetics

View full text Add to dashboard Cite

PurposeGrid computing is gaining more significance in the high‐performance computing world. This concept leads to the discovery of solutions for complicated problems regarding the diversity of available resources among different jobs in the grid. However, the major problem is the optimal job scheduling for heterogeneous resources, in which each job needs to be allocated to a proper grid's node with the appropriate resources. An important challenge is to solve optimally the scheduling problem, because the capability and availability of resources vary dynamically and the complexity of scheduling increases with the size of the grid. The purpose of this paper is to present a framework which combines the fuzzy C‐mean (FCM) clustering with an ant colony optimization (ACO) algorithm to improve the scheduling decision when the grid is heterogeneous.Design/methodology/approachIn the proposed model, the FCM algorithm classifies the jobs into appropriate classes, and the ACO algorithm maps the jobs to the appropriate resources. The ACO is characterized by ant‐like mobile agents that cooperate and stochastically explore a network, iteratively building solutions based on their own memory and on the traces (pheromone levels) left by other agents.FindingsThe simulation is done by using historical information on jobs in a grid. The experimental results show that the proposed algorithm can allocate jobs more efficiently and more effectively than the traditional algorithms for scheduling policies.Originality/valueThe paper provides a scheduling model based on FCM clustering and ACO algorithm for grid scheduling. The authors compared the performance of the proposed algorithm with the performance of various job‐scheduling algorithms in the grid computing environment. The comparison results show that the proposed algorithm outperforms other algorithms and gives optimal results.

show abstract

Section: Literature Reviewmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Colony of cooperating agents based independent job scheduling in a computation grid

Helmy

Rasheed

2011

International Journal of Intelligent Computing and Cybernetics

View full text Add to dashboard Cite

show abstract

“…Since the mid-1990s huge strides made in this area, and implementations such as Xen can allow a single server to run dozens of virtual machines with minimal overhead [110,111]. In fact, studies even suggest that thousands of virtual machines can be simultaneously launched across a cluster with little difficulty [112]. Consequently, there has been a movement toward server consolidation in the data center to minimize idle resources and their associated costs [113].…”

Section: Virtual Machine Technologymentioning

confidence: 99%

Dynamic Fractional Resource Scheduling for cluster platforms

Stillwell

2010

2010 IEEE International Symposium on Parallel &Amp; Distributed Processing, Workshops and PHD Forum (IPDPSW)

View full text Add to dashboard Cite

Experiments presented in this paper were carried out using the Grid'5000 experimental testbed, being developed under the INRIA ALADDIN development action with support from CNRS, RENATER and several Universities as well as other funding bodies (see https://www.grid5000.fr).The experiments using our prototype system were carried out on machines owned by the Deparment of Computer Science and Engineering at the University of California, San Diego with the permission of Geoffrey Voelker. System administration and much assistance were provided by Gjergji Zyba. iv ABSTRACTThis research focuses on the problem of job scheduling on homogeneous computational clusters. Clusters are widely used today for a variety of purposes, including high-performance scientific computing and Internet service hosting. While clusters may have impressive aggregate performance metrics, they are really only collections of fairly modest machines, which makes scheduling jobs for the best performance a non-trivial problem. Most clusters also need to be shared among users to amortize their start-up and maintenance costs, and ensuring that these users are treated fairly further adds to the difficulty. Existing approaches to scheduling attempt to address both of these issues, but have several limitations.We propose a novel approach, called Dynamic Fractional Resource Scheduling (DFRS), to sharing homogeneous cluster computing platforms among competing jobs.The key features of DFRS are that it leverages existing virtual machine technology in order to share resources more efficiently and it defines and optimizes a user-centric metric that captures notions of both performance and fairness. In this dissertation we explain the principles behind DFRS and its advantages over the current state of the art, develop a theoretical model of resource sharing, design heuristics to optimize the proposed metric within the given framework, implement and run simulations comparing DFRS to traditional approaches using popular and accepted performance metrics, and finally develop and test a prototype implementation based on existing technologies. Our results show that it is possible to develop heuristic algorithms that give results reasonably close to theoretical bounds for a variety of cases, that resource requirements are well within the capabilities of modern systems, and that for some scenarios DFRS can provide orders-ofmagnitude levels of improvement in performance over current approaches.

show abstract