HPC resource integration into CMS Computing via HEPCloud

Hufnagel, D.; Holzman, B.; Mason, D.; Mhashilkar, Parag; Timm, S.; Tiradani, Anthony; Khan, Farrukh Aftab; Gutsche, O.; Bloom, K.

doi:10.1051/epjconf/201921403031

Cited by 5 publications

(4 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The HPC resources located in the US are integrated into the CMS systems via HEPCloud [355], a portal to an ecosystem of diverse computing resources, commercial or academic, hosted at FNAL. Since 2020, HEPCloud has provisioned resources from seven different HPC centers located for example at the NERSC, PSC, TACC, or SDSC computing centers.…”

Section: High Performance Computing (Hpc)mentioning

confidence: 99%

Development of the CMS detector for the CERN LHC Run 3

Hayrapetyan,

Tumasyan,

Adam

et al. 2024

J. Inst.

View full text Add to dashboard Cite

Since the initial data taking of the CERN LHC, the CMS experiment has undergone substantial upgrades and improvements. This paper discusses the CMS detector as it is configured for the third data-taking period of the CERN LHC, Run 3, which started in 2022. The entire silicon pixel tracking detector was replaced. A new powering system for the superconducting solenoid was installed. The electronics of the hadron calorimeter was upgraded. All the muon electronic systems were upgraded, and new muon detector stations were added, including a gas electron multiplier detector. The precision proton spectrometer was upgraded. The dedicated luminosity detectors and the beam loss monitor were refurbished. Substantial improvements to the trigger, data acquisition, software, and computing systems were also implemented, including a new hybrid CPU/GPU farm for the high-level trigger.

show abstract

Section: High Performance Computing (Hpc)mentioning

confidence: 99%

Development of the CMS detector for the CERN LHC Run 3

Hayrapetyan,

Tumasyan,

Adam

et al. 2024

J. Inst.

View full text Add to dashboard Cite

show abstract

“…In certain cases HPC centers do not allow outgoing connectivity from the compute nodes, effectively limiting the potential resources that CMS can integrate using standard mechanisms. Multiple R&D efforts have been performed in CMS in order to overcome these limitations [7][8][9]. Some of these rely on the existence of edge services or special nodes at the HPC center that can bridge network connectivity to the outer world.…”

Section: Network Requirements For Hpc Resource Exploitation By Cmsmentioning

confidence: 99%

Integration of the Barcelona Supercomputing Center for CMS computing: Towards large scale production

Acosta-Silva,

Delgado Peris,

Flix Molina

et al. 2024

EPJ Web of Conf.

View full text Add to dashboard Cite

The CMS experiment is working to integrate an increasing number of High Performance Computing (HPC) resources into its distributed computing infrastructure. The case of the Barcelona Supercomputing Center (BSC) is particularly challenging as severe network restrictions prevent the use of CMS standard computing solutions. The CIEMAT CMS group has performed significant work in order to overcome these constraints and make BSC resources available to CMS. The developments include adapting the workload management tools, replicating the CMS software repository to BSC storage, providing an alternative access to detector conditions data, and setting up a service to transfer produced output data to a nearby storage facility. In this work, we discuss the current status of this integration activity and present recent developments, such as a front-end service to improve slot usage efficiency and an enhanced transfer service that supports the staging of input data for workflows at BSC. Moreover, significant efforts have been devoted to improving the scalability of the deployed solution, automating its operation, and simplifying the matchmaking of CMS workflows that are suitable for execution at BSC.

show abstract

“…As shown in Figure 2, CMS follows two main models for the integration of HPC resources to our computing resource portfolio: firstly, in the US, the HEPCloud [10] infrastructure, managed by the FNAL team, supports a separated HTCondor pool, which is however federated to the central CMS SI, being delegated the execution of CMS tasks [11]. Secondly, a method of WLCG site-extension (transparent from the CMS perspective) is mainly employed for the EU HPC facilities [12] [13].…”

Section: Hpc Growing Contribution To Cms Computingmentioning

confidence: 99%

HPC resources for CMS offline computing: An integration and scalability challenge for the Submission Infrastructure

Pérez-Calero Yzquierdo,

Mascheroni,

Kizinevic

et al. 2024

EPJ Web of Conf.

View full text Add to dashboard Cite

The computing resource needs of LHC experiments are expected to continue growing significantly during the Run 3 and into the HL-LHC era. The landscape of available resources will also evolve, as High Performance Computing (HPC) and Cloud resources will provide a comparable, or even dominant, fraction of the total compute capacity. The future years present a challenge for the experiments’ resource provisioning models, both in terms of scalability and increasing complexity. The CMS Submission Infrastructure (SI) provisions computing resources for CMS workflows. This infrastructure is built on a set of federated HTCondor pools, currently aggregating 400k CPU cores distributed worldwide and supporting the simultaneous execution of over 200k computing tasks. Incorporating HPC resources into CMS computing represents firstly an integration challenge, as HPC centers are much more diverse compared to Grid sites. Secondly, evolving the present SI, dimensioned to harness the current CMS computing capacity, to reach the resource scales required for the HLLHC phase, while maintaining global flexibility and efficiency, will represent an additional challenge for the SI. To preventively address future potential scalability limits, the SI team regularly runs tests to explore the maximum reach of our infrastructure. In this note, the integration of HPC resources into CMS offline computing is summarized, the potential concerns for the SI derived from the increased scale of operations are described, and the most recent results of scalability test on the CMS SI are reported.

show abstract

HPC resource integration into CMS Computing via HEPCloud

Cited by 5 publications

References 7 publications

Development of the CMS detector for the CERN LHC Run 3

Development of the CMS detector for the CERN LHC Run 3

Integration of the Barcelona Supercomputing Center for CMS computing: Towards large scale production

HPC resources for CMS offline computing: An integration and scalability challenge for the Submission Infrastructure

Contact Info

Product

Resources

About