Context-aware distributed cloud computing using CloudScheduler

Seuster, R.; Leavett-Brown, CR; Casteels, Kevin; Driemel, Colson; Paterson, Michael; Ring, Dominic; Sobie, R.; Taylor, R.; Weldon, J.

doi:10.1088/1742-6596/898/5/052039

Cited by 1 publication

(1 citation statement)

References 1 publication

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Such compute clouds can be research clouds located at a university or computing center, or commercial clouds that can be located anywhere in the world. At the University of Victoria (UVic), we use a system called cloudscheduler [1] to integrate the distributed cloud resources as worker nodes of a HTCondor batch system [2].…”

Section: Introductionmentioning

confidence: 99%

Using a dynamic data federation for running Belle-II simulation applications in a distributed cloud environment

et al. 2019

View full text Add to dashboard Cite

The dynamic data federation software Dynafed, developed by CERN IT, provides a federated storage cluster on demand using the HTTP protocol with WebDAV extensions. Traditional storage sites which support an experiment can be added to Dynafed without requiring any changes to the site. Dynafed also supports direct access to cloud storage such as S3 and Azure. We report on the usage of Dynafed to support Belle-II production jobs running on a distributed cloud system utilizing clouds across North America. Cloudscheduler, developed by the University of Victoria HEP Research Computing group , federates Openstack, OpenNebula, Amazon, Google, and Microsoft cloud compute resources and provides them as a unified Grid site which on average runs about 3500 Belle-II production jobs in parallel. The input data for those jobs is accessible through a single endpoint, our Dynafed instance. This Dynafed instance unifies storage resources provided by Amazon S3, Ceph, and Minio object stores as endpoints, as well as storage provided by traditional DPM and dCache sites. We report on our long term experience with this setup, the implementation of a grid-mapfile based X509 authentication/authorization for Belle-II access, and we show how a federated cluster can be used by Belle-II through

show abstract

Section: Introductionmentioning

confidence: 99%