Utilizing clouds for Belle II

Sobie, R.

doi:10.1088/1742-6596/664/2/022037

Cited by 4 publications

(5 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The use of cloud resources in scientific collaborations is starting to be more and more common [8] [9]. In order to be prepared to efficiently benefit from cloud resources that may be available to the CTA Consortium, we have tested their integration in the CTA-DIRAC setup.…”

Section: Cloud Resources Integrationmentioning

confidence: 99%

The Cherenkov Telescope Array production system for data-processing and Monte Carlo simulation

et al. 2019

View full text Add to dashboard Cite

The Cherenkov Telescope Array (CTA) is the next-generation instrument in the field of very high energy gamma-ray astronomy. It will be composed of two arrays of Imaging Atmospheric Cherenkov Telescopes, located at La Palma (Spain) and Paranal (Chile). The construction of CTA has just started with the installation of the first telescope on site at La Palma and the first data expected by the end of 2018. The scientific operations should begin in 2022 for a duration of about 30 years. The overall amount of data produced during these operations is around 27 PB per year. The associated computing power for data processing and Monte Carlo (MC) simulations is of the order of hundreds of millions of CPU HS06 hours per year. In order to cope with these high computing requirements, we have developed a production system prototype based on the DIRAC framework, that we have intensively exploited during the past 6 years to handle massive MC simulations on the grid for the CTA design and prototyping phases. CTA workflows are composed of several inter-dependent steps, which we used to handle separately within our production system. In order to fully automatize the whole workflows execution, we have partially revised the production system by further enhancing the data-driven behavior and by extending the use of meta-data to link together the different steps of a workflow. In this contribution we present the application of the production system to the last years MC campaigns as well as the recent production system evolution, intended to obtain a fully data-driven and automatized workflow execution for efficient processing of real telescope data.

show abstract

Section: Cloud Resources Integrationmentioning

confidence: 99%

The Cherenkov Telescope Array production system for data-processing and Monte Carlo simulation

et al. 2019

View full text Add to dashboard Cite

show abstract

“…The design and implementation of the systems is usually driven by the requirement to integrate the cloud services into an existing local infrastructure. For example, the Belle II experiment at the KEK Laboratory in Japan uses four different methods to exploit clouds [4]. This paper will focus on our system that integrates private and opportunistic clouds into a single batch computing environment [5].…”

Section: Distributed Cloud Computingmentioning

confidence: 99%

“…Cloud computing is having a significant impact in the field of high-energy physics (HEP) [1,2,3,4]. One can attribute this, in part, to the widespread adoption and investment in cloud computing by the commercial world.…”

Section: Introductionmentioning

confidence: 99%

Context-aware cloud computing for HEP

Sobie¹

2017

Proceedings of International Symposium on Grids and Clouds (ISGC) 2016 — PoS(ISGC 2016)

View full text Add to dashboard Cite

The use of cloud computing is increasing in the field of high-energy physics (HEP). Both dedicated and opportunistic cloud resources are delivering a growing fraction of the requirements of large projects. The use of context-aware designs, such as those used in the mobile communication industry, can improve our distributed systems and make them more responsive to changing demands and dynamic resources. We show how context-aware service have been implemented and their impact on the system. We describe our plans to add new context-aware capabilities in the future.

show abstract

“…The original design of cloudscheduler was conceived in 2009 [2] and is based on ideas discussed in a paper describing "sky computing" [3]. The distributed compute cloud has run many millions of jobs on three continents for the ATLAS experiment at the CERN Laboratory in Geneva, Switzerland [4] and the Belle II experiment at the KEK Laboratory in Tsukuba, Japan [5].…”

Section: Introductionmentioning

confidence: 99%

High-Throughput Cloud Computing with the Cloudscheduler VM Provisioning Service

Berghaus

Casteels

Driemel

et al. 2020

Comput Softw Big Sci

View full text Add to dashboard Cite

We describe a high-throughput computing system for running jobs on public and private computing clouds using the HTCondor job scheduler and the cloudscheduler VM provisioning service. The distributed cloud computing system is designed to simultaneously use dedicated and opportunistic cloud resources at local and remote locations. It has been used for large-scale production particle physics workloads for many years using thousands of cores on three continents. A decade after its initial design and implementation, cloudscheduler has been modernized to take advantage of new software designs, improved operating system capabilities and support packages. The updated cloudscheduler is more resilient and scalable, with expanded capabilities. We present an overview of the original design and then describe the new version of the distributed compute cloud system. We conclude with a review of the current status and future plans.

show abstract

Utilizing clouds for Belle II

Cited by 4 publications

References 8 publications

The Cherenkov Telescope Array production system for data-processing and Monte Carlo simulation

The Cherenkov Telescope Array production system for data-processing and Monte Carlo simulation

Context-aware cloud computing for HEP

High-Throughput Cloud Computing with the Cloudscheduler VM Provisioning Service

Contact Info

Product

Resources

About