A Meta-Brokering Framework for Science Gateways

Abstract

Recently scientific communities produce a growing number of computation-intensive applications, which calls for the interoperation of distributed infrastructures including Clouds, Grids and private clusters. The European SHIWA and ER-flow projects have enabled the combination of heterogeneous scientific workflows, and their execution in a large-scale system consisting of multiple Distributed Computing Infrastructures. One of the resource management challenges of these projects is called parameter study job scheduling. A parameter study job of a workflow generally has a large number of input files to be consumed by independent job instances. In this paper we propose a meta-brokering framework for science gateways to support the execution of such workflows. In order to cope with the high uncertainty and unpredictable load of the utilized distributed infrastructures, we introduce the so called resource priority services. These tools are capable of determining and dynamically updating priorities of the available infrastructures to be selected for job instances. Our evaluations show that this approach implies an efficient distribution of job

Selection of computational environments for PSP processing on scientific gateways

“…Karoczkai et al [56] presented a meta-broker for science gateways for scheduling jobs to heterogeneous machines. It creates a layer on the gUSE gateway to set job priorities and distributes them according to the distance from of the service providers.…”

Science Gateway Workshops 2015 Special Issue Conference Publications

“…The work of Karoczkai, Kertesz and Kacsuk in "A Meta-Brokering Framework for Science Gateways" [12] focuses on a more specific problem in resource management for for science gateways developed for the gUSE family. Their metabrokering framework solves this issue through resource priority services, and the concept and services can be re-used for science gateways supporting the BES standard.…”