CernVM-FS – beyond LHC computing

Condurache, Catalin; Collier, Ian

doi:10.1088/1742-6596/513/3/032020

Cited by 5 publications

(4 citation statements)

References 1 publication

Supporting

Mentioning

Contrasting

Order By: Relevance

“…CernVM-FS is a read-only, globally distributed filesystem that is optimized for distributing software. 30,31 It was developed in the context of the Large Hadron Collider (LHC) research project 32 and the Worldwide LHC Computing Grid (WLCG), * to efficiently distribute application software across LHC project members worldwide. A CernVM-FS repository contains actual software installations, not packages that contain software like a traditional package management repository we discussed in Section 2.2.…”

Section: Cernvm-fsmentioning

confidence: 99%

“…In the context of grid computing (where the compute resources are widely distributed), CernVM‐FS (CernVM File System) is a common tool used in particular by the high‐energy physics community. CernVM‐FS is a read‐only, globally distributed filesystem that is optimized for distributing software 30,31 . It was developed in the context of the Large Hadron Collider (LHC) research project 32 and the Worldwide LHC Computing Grid (WLCG),to efficiently distribute application software across LHC project members worldwide.…”

Section: Current Practicementioning

confidence: 99%

See 1 more Smart Citation

EESSI: A cross‐platform ready‐to‐use optimised scientific software stack

et al. 2022

View full text Add to dashboard Cite

Getting scientific software installed correctly and ensuring it performs well has been a ubiquitous problem for several decades now, which is compounded currently by the changing landscape of computational science with the (re‐)emergence of different microprocessor families, and the expansion to additional scientific domains like artificial intelligence and next‐generation sequencing. The European Environment for Scientific Software Installations (EESSI) project aims to provide a ready‐to‐use stack of scientific software installations that can be leveraged easily on a variety of platforms, ranging from personal workstations to cloud environments and supercomputer infrastructure, without making compromises with respect to performance. In this article, we provide a detailed overview of the project, highlight potential use cases, and demonstrate that the performance of the provided scientific software installations can be competitive with system‐specific installations.

show abstract

Section: Cernvm-fsmentioning

confidence: 99%

Section: Current Practicementioning

confidence: 99%

EESSI: A cross‐platform ready‐to‐use optimised scientific software stack

et al. 2022

View full text Add to dashboard Cite

show abstract

“…The Docker image is based on the distribution from IHEP. CernVM File System (CVMFS) [9] is configured by default in the images, so users could access the JUNO software on-demand.…”

Section: Infrastructuresmentioning

confidence: 99%

Jupyter-based service for JUNO analysis

Lin

2020

EPJ Web Conf.

View full text Add to dashboard Cite

The JUNO (Jiangmen Underground Neutrino Observatory) is designed to determine the neutrino mass hierarchy and precisely measure oscillation parameters. The estimated data volume of raw data is about 2 PB/year. The event rate of reactor anti-neutrinos is about 60/day, while the event rate of background is about O(10) Hz. The challenge is the event correlation during the analysis, where the background events could not be discarded. In order to use big data techniques to search for rare events, a Jupyter-based interactive service is developed for JUNO analysis. In this paper, an overview of this service is presented. The infrastructure is based on Jupyter and Kubernetes, which provides the user interface and resource management. In order to integrate the data processing framework and big data techniques, an index file is used as an intermediate file, which points to the interested events. Data processing framework SNiPER is used to select the candidate of neutrino signals and produce the index file. Apache Spark is then used to process such index file repeatedly with data cached in memory. With the index file produced from Spark and the complete event data files, SNiPER is used to process them and produce the final physics result. At the end of paper, the test-bed is presented and the testing result is shown.

show abstract

“…For the LHC experiments with their world-wide distributed grid computing infrastructure, we managed to reduce the time between building a new software release and the rollout from several days to less than one hour. As a result there is now a big increase in the number of scientific users in high-energy physics and in various other fields [7], [8].…”

Section: A Time Machine For the Data Analysis Environmentmentioning

confidence: 99%

The Need for a Versioned Data Analysis Software Environment

Blomer,

Berzano,

Buncic

et al. 2014

Preprint

View full text Add to dashboard Cite

Scientific results in high-energy physics and in many other fields often rely on complex software stacks. In order to support reproducibility and scrutiny of the results, it is good practice to use open source software and to cite software packages and versions. With evergrowing complexity of scientific software on one side and with IT life-cycles of only a few years on the other side, however, it turns out that despite source code availability the setup and the validation of a minimal usable analysis environment can easily become prohibitively expensive. We argue that there is a substantial gap between merely having access to versioned source code and the ability to create a data analysis runtime environment. In order to preserve all the different variants of the data analysis runtime environment, we developed a snapshotting file system optimized for software distribution. We report on our experience in preserving the analysis environment for high-energy physics such as the software landscape used to discover the Higgs boson at the Large Hadron Collider.

show abstract

CernVM-FS – beyond LHC computing

Cited by 5 publications

References 1 publication

EESSI: A cross‐platform ready‐to‐use optimised scientific software stack

EESSI: A cross‐platform ready‐to‐use optimised scientific software stack

Jupyter-based service for JUNO analysis

The Need for a Versioned Data Analysis Software Environment

Contact Info

Product

Resources

About