Intelligent job selection for distributed scheduling

Abstract

A key issue in distributed scheduling is selecting appropriate jobs to transfer. In this paper, a job selection policy that considers the diversity of job behaviors is proposed. A mechanism used in artifrcial neural networks, called weight climbing, is employed. Using this mechanism, a distributed scheduler can learn the behavior of a job from its past executions and make a correct prediction about whether transferring the job is worthwhile. A scheduler using the proposed job selection policy has been implemented and experimental results show that it is able to learn job behaviors fast, make decisions accurately and adjust itself promptly when system conjguration or program behaviors are changed. In addition, the selection policy introduces only negligible time and space overhead.*ReseaKh r e P 0~ hereh was supported by 'he National science Foundation under grant No. CCR-8909072 and by U.S. Amy Research Office under contraas No. DAAU3-91-G-0093 and NO. DAAL03-92 ory hungry job might prefer a large amount of memory over a lightly loaded CPU, and an interactive job, which G-0184. n e views, opinions, and/or findings contained in this paper are those of the authors and should not be construed as an official Department of the Army position, policy or decision.constantly needs YO through keyboards and screens, will probably run better locally. Therefore, a selection policy 517 0-8186-3770-6/93 $3.00 0 1993 IEEE