CERN Computing in Commercial Clouds

Cordeiro, Cristovao; Field, Laurence; Bear, Borja Garrido; Giordano, D.; Jones, Ben; Keeble, Oliver; Manzi, Andrea; Martelli, Edoardo; McCance, Gavin; Moreno-Garcia, Diana; Traylen, S.

doi:10.1088/1742-6596/898/8/082030

Cited by 5 publications

(4 citation statements)

References 3 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As described in section 12.7.1, the SI computing capacity has recently expanded into HPC facilities which are integrated as part of the Global or HEPCloud pools. CERN on-site CMS resources, along with opportunistic local (BEER [375]) and cloud [376,377] computing slots, are organized into a third HTCondor pool, built on a dedicated set of hosts to isolate it from potential issues in the main Global Pool, given its critical role in supporting Tier 0 tasks during data-taking periods. Specialized nodes known as "schedds" control workload submission.…”

Section: Central Processing and Productionmentioning

confidence: 99%

Development of the CMS detector for the CERN LHC Run 3

Hayrapetyan,

Tumasyan,

Adam

et al. 2024

J. Inst.

View full text Add to dashboard Cite

Since the initial data taking of the CERN LHC, the CMS experiment has undergone substantial upgrades and improvements. This paper discusses the CMS detector as it is configured for the third data-taking period of the CERN LHC, Run 3, which started in 2022. The entire silicon pixel tracking detector was replaced. A new powering system for the superconducting solenoid was installed. The electronics of the hadron calorimeter was upgraded. All the muon electronic systems were upgraded, and new muon detector stations were added, including a gas electron multiplier detector. The precision proton spectrometer was upgraded. The dedicated luminosity detectors and the beam loss monitor were refurbished. Substantial improvements to the trigger, data acquisition, software, and computing systems were also implemented, including a new hybrid CPU/GPU farm for the high-level trigger.

show abstract

Section: Central Processing and Productionmentioning

confidence: 99%

Development of the CMS detector for the CERN LHC Run 3

Hayrapetyan,

Tumasyan,

Adam

et al. 2024

J. Inst.

View full text Add to dashboard Cite

show abstract

“…With the planned upgrades of the LHC and expected growth in the amount and complexity of data collected by its experiments, CERN’s computing infrastructures will be facing a large and challenging demand of computing resources (CERN Tape Archive) (Fartoukh et al, 2021). Within this scope, the adoption of OpenStack cloud computing at CERN has opened the door to the evaluation of commercial cloud services, which could supply additional Infrastructure-as-a-Service (IaaS) resources to extend the current CERN computing resources for physics data processing (Cordeiro et al, 2017). While OpenStack is highly customisable for cloud-native workloads, it has not been adopted in HPC, especially for large, multi-Petascale systems running MPI applications at scale.…”

Section: Related Workmentioning

confidence: 99%

Versatile software-defined HPC and cloud clusters on Alps supercomputer for diverse workflows

Alam

Gila

Klein

et al. 2023

The International Journal of High Performance Computing Applica

View full text Add to dashboard Cite

Supercomputers have been driving innovations for performance and scaling benefiting several scientific applications for the past few decades. Yet their ecosystems remain virtually unchanged when it comes to integrating distributed data-driven workflows, primarily due to rather rigid access methods and restricted configuration management options. X-as-a-Service model of cloud has introduced, among other features, a developer-centric DevOps approach empowering developers of infrastructure, platform to software artefacts, which, unfortunately contemporary supercomputers still lack. We introduce vClusters (versatile software-defined clusters), which is based on Infrastructure-as-code (IaC) technology. vClusters approach is a unique fusion of HPC and cloud technologies resulting in a software-defined, multi-tenant cluster on a supercomputing ecosystem, that, together with software-defined storage, enable DevOps for complex, data-driven workflows like grid middleware, alongside a classic HPC platform. IaC has been a commonplace in cloud computing, however, it lacked adoption within multi-Petascale ecosystems due to concerns related to performance and interoperability with classic HPC data centres’ ecosystems. We present an overview of the Swiss National Supercomputing Centre’s flagship Alps ecosystem as an implementation target for vClusters for HPC and data-driven workflows. Alps is based on the Cray-HPE Shasta EX supercomputing platform that includes an IaC compliant, microservices architecture (MSA) management system, which we leverage for demonstrating vClusters usage for our diverse operational workflows. We provide implementation details of two operational vClusters platforms: a classic HPC platform that is used predominantly by hundreds of users running thousands of large-scale numerical simulations batch jobs; and a widely used, data-intensive, Grid computing middleware platform used for CERN Worldwide LHC Computing Grid (WLCG) operations. The resulting solution showcases reuse and reduction of common configuration recipes across vCluster implementations, minimising operational change management overheads while introducing flexibility for managing artefacts for DevOps required by diverse workflows.

show abstract

“…In the context of the IT department, the infrastructure functionality has grown during these years, from container orchestration engines with OpenStack Magnum [4,5] to ()api for baremetal provisioning with OpenStack Ironic [6,7]. Finally, the Batch Service has faced not only the challenge of scaling-up the pool but to integrate opportunistic resources [8] and to also explore the exploitation of external clouds to expand capacity [9,10]. In a rapidly changing environment, some constants remain: the compute demand keeps growing, and the IT budget is still tight.…”

Section: Introductionmentioning

confidence: 99%

Managing the CERN Batch System with Kubernetes

et al. 2020

View full text Add to dashboard Cite

The CERN Batch Service faces many challenges in order to get ready for the computing demands of future LHC runs. These challenges require that we look at all potential resources, assessing how efficiently we use them and that we explore different alternatives to exploit opportunistic resources in our infrastructure as well as outside of the CERN computing centre. Several projects, like BEER, Helix Nebula Science Cloud and the new OCRE project, have proven our ability to run batch workloads on a wide range of non-traditional resources. However, the challenge is not only to obtain the raw compute resources needed but how to define an operational model that is cost and time efficient, scalable and flexible enough to adapt to a heterogeneous infrastructure. In order to tackle both the provisioning and operational challenges it was decided to use Kubernetes. By using Kubernetes we benefit from a de-facto standard in containerised environments, available in nearly all cloud providers and surrounded by a vibrant ecosystem of open-source projects. Leveraging Kubernetes’ built-in functionality, and other open-source tools such as Helm, Terraform and GitLab CI, we have deployed a first cluster prototype which we discuss in detail. The effort has simplified many of the existing operational procedures we currently have, but has also made us rethink established procedures and assumptions that were only valid in a VM-based cloud environment. This contribution presents how we have adopted Kubernetes into the CERN Batch Service, the impact its adoption has in daily operations, a comparison on resource usage efficiency and the experience so far evolving our infrastructure towards this model.

show abstract

CERN Computing in Commercial Clouds

Cited by 5 publications

References 3 publications

Development of the CMS detector for the CERN LHC Run 3

Development of the CMS detector for the CERN LHC Run 3

Versatile software-defined HPC and cloud clusters on Alps supercomputer for diverse workflows

Managing the CERN Batch System with Kubernetes

Contact Info

Product

Resources

About