GWpilot: Enabling multi-level scheduling in distributed infrastructures with GridWay and pilot jobs

Montero, A. J. Rubio; Huedo, Eduardo; Castejón, F.; Mayo-García, Rafael

doi:10.1016/j.future.2014.10.003

Cited by 21 publications

(26 citation statements)

References 67 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The Grid implementation of CORSIKA was deployed with two “flavors,” which run by using GridWay Metascheduler (http://www.gridway.org/doku.php; Huedo et al, )or with a second approach through a Catania Science Gateway interface (Barbera et al, ). Massive calculations can be executed with the former, via the Montera (Rodriguez‐Pascual et al, ), the GWpilot (Rubio‐Montero, Huedo, Castejón, & Mayo‐García, ), or the GWcloud (Rubio‐Montero, Huedo, & Mayo‐García, ) frameworks.…”

Section: The Space Weather Simulation Chainmentioning

confidence: 99%

Preliminary Results From the Latin American Giant Observatory Space Weather Simulation Chain

2018

View full text Add to dashboard Cite

The Space Weather program of the Latin American Giant Observatory (LAGO) Collaboration was designed to study the variation of the flux of atmospheric secondary particles at ground level produced during the interaction of cosmic rays with the air. This work complements and expands the inference capabilities of the LAGO detection network to identify the influence of solar activity on the particle flux, at places having different geomagnetic rigidity cut‐offs and atmospheric depths. This program is developed through a series of Monte Carlo sequential simulations to compute the intensity spectrum of the various components of the radiation field on the ground. A key feature of these calculations is that we performed detailed radiation transport computations as a function of incident direction, time, altitude, and latitude and longitude. Magnetic rigidity calculations and corrections for geomagnetic field activity are established by using the MAGNETOCOSMICS code, and the estimation of the flux of secondaries at ground level is implemented by using the CORSIKA code; thus, we can examine the local peculiarities in the penumbral regions with a more realistic description of the atmospheric and geomagnetic response in these complex regions of the rigidity space. As an example of our calculation scheme, we report some result on the flux at ground level for two LAGO locations: Bucaramanga, Colombia, and San Carlos de Bariloche, Argentina, for the geomagnetically active period of May 2005.

show abstract

Section: The Space Weather Simulation Chainmentioning

confidence: 99%

Preliminary Results From the Latin American Giant Observatory Space Weather Simulation Chain

2018

View full text Add to dashboard Cite

show abstract

“…The Grid implementation of CORSIKA was deployed in two 'flavors' being able to run by using GridWay Metascheduler 2 [9] or with a second approach through a Catania Science Gateway interface [10]. For the former, massive calculations can be executed via the Montera [11], the GWpilot [12] or the GWcloud [13] frameworks In the Science Gateway approach a user can seamlessly run a code on different infrastructures by accessing a unique entry point with an identity provision. He/she only has to upload the input data or use a PID to reference it and click on the run icon.…”

Section: Lago Simulationsmentioning

confidence: 99%

The Latin American Giant Observatory: A Successful Collaboration in Latin America Based on Cosmic Rays and Computer Science Domains

Asorey

Núñez

Suárez-Durán

et al. 2016

2016 16th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid)

Self Cite

View full text Add to dashboard Cite

In this work the strategy of the Latin American Giant Observatory (LAGO) to build a Latin American collaboration is presented. Installing Cosmic Rays detectors settled all around the Continent, from Mexico to the Antarctica, this collaboration is forming a community that embraces both high energy physicist and computer scientists. This is so because the data that are measured must be analytical processed and due to the fact that a priori and a posteriori simulations representing the effects of the radiation must be performed. To perform the calculi, customized codes have been implemented by the collaboration. With regard to the huge amount of data emerging from this network of sensors and from the computational simulations performed in a diversity of computing architectures and e-infrastructures, an effort is being carried out to catalog and preserve a vast amount of data produced by the waterCherenkov Detector network and the complete LAGO simulation workflow that characterize each site. Metadata, Permanent Identifiers and the facilities from the LAGO Data Repository are described in this work jointly with the simulation codes used. These initiatives allow researchers to produce and find data and to directly use them in a code running by means of a Science Gateway that provides access to different clusters, Grid and Cloud infrastructures worldwide.

show abstract

“…feed the actual resource with own jobs. For example DIRAC and GWpilot [4] use this approach. Concerning the more traditional push approach, generally all job instances are submitted simultaneously to the scheduling component of the workflow management system.…”

Section: Related Workmentioning

confidence: 99%

A Meta-Brokering Framework for Science Gateways

2016

View full text Add to dashboard Cite

Recently scientific communities produce a growing number of computation-intensive applications, which calls for the interoperation of distributed infrastructures including Clouds, Grids and private clusters. The European SHIWA and ER-flow projects have enabled the combination of heterogeneous scientific workflows, and their execution in a large-scale system consisting of multiple Distributed Computing Infrastructures. One of the resource management challenges of these projects is called parameter study job scheduling. A parameter study job of a workflow generally has a large number of input files to be consumed by independent job instances. In this paper we propose a meta-brokering framework for science gateways to support the execution of such workflows. In order to cope with the high uncertainty and unpredictable load of the utilized distributed infrastructures, we introduce the so called resource priority services. These tools are capable of determining and dynamically updating priorities of the available infrastructures to be selected for job instances. Our evaluations show that this approach implies an efficient distribution of job

show abstract

GWpilot: Enabling multi-level scheduling in distributed infrastructures with GridWay and pilot jobs

Cited by 21 publications

References 67 publications

Preliminary Results From the Latin American Giant Observatory Space Weather Simulation Chain

Preliminary Results From the Latin American Giant Observatory Space Weather Simulation Chain

The Latin American Giant Observatory: A Successful Collaboration in Latin America Based on Cosmic Rays and Computer Science Domains

A Meta-Brokering Framework for Science Gateways

Contact Info

Product

Resources

About