The XENON1T Data Distribution and Processing Scheme

Abstract

The XENON experiment is looking for non-baryonic particle dark matter in the universe. The setup is a dual phase time projection chamber (TPC) filled with 3200 kg of ultra-pure liquid xenon. The setup is operated at the Laboratori Nazionali del Gran Sasso (LNGS) in Italy. We present a full overview of the computing scheme for data distribution and job management in XENON1T. The software package Rucio, which is developed by the ATLAS collaboration, facilitates data handling on Open Science Grid (OSG) and European Grid Infrastructure (EGI) storage systems. A tape copy at the Centre for High Performance Computing (PDC) is managed by the Tivoli Storage Manager (TSM). Data reduction and Monte Carlo production are handled by CI Connect which is integrated into the OSG network. The job submission system connects resources at the EGI, OSG, SDSC's Comet, and the campus HPC resources for distributed computing.

Dark-matter And Neutrino Computation Explored (DANCE) Community Input to Snowmass

“…Experiments are generating increasing amounts of data -on the order of petabytes (PBs) per year for the upcoming experiments -and this requires coordinated data management systems that will scale. There are tools that manage data on this scale, but all of these require effort to adapt these tools to the needs of our community and our specific experiments [12]. Support is needed to adapt these tools to new scientific toolboxes within experiments and related computing clusters (to meet the requirements of the software without adding security risk).…”

Rucio beyond ATLAS: experiences from Belle II, CMS, DUNE, EISCAT3D, LIGO/VIRGO, SKA, XENON

“…XENON1T has taken more than 800 TB of raw data and ran multiple re-processing campaigns for improving data quality in ongoing data analysis tasks. [25] The upcoming XENONnT upgrade will take 1 petabyte per year. Processing and Monte Carlo simulation campaigns are planned at the major infrastructures EGI an OSG.…”