ATLAS Global Shares implementation in PanDA

Abstract

PanDA (Production and Distributed Analysis) is the workload management system for ATLAS across the Worldwide LHC Computing Grid. While analysis tasks are submitted to PanDA by over a thousand users following personal schedules (e.g. PhD or conference deadlines), production campaigns are scheduled by a central Physics Coordination group based on the organization's calendar. The Physics Coordination group needs to allocate the amount of Grid resources dedicated to each activity, in order to manage sharing of CPU resources among various parallel campaigns and to make sure that results can be achieved in time for important deadlines. While dynamic and static shares on batch systems have been around for a long time, we are trying to move away from local resource partitioning and manage shares at a global level in the PanDA system. The global solution is not straightforward, given different requirements of the activities (number of cores, memory, I/O and CPU intensity), the heterogeneity of Grid resources (site/HW capabilities, batch configuration and queue setup) and constraints on data locality. We have therefore started the Global Shares project that follows a requirements-driven multi-step execution plan, starting from definition of nestable shares, implementing share-aware job dispatch, aligning internal processes with global shares and finally implementing a pilot stream control for controlling the batch slots that keeps late binding. This contribution will explain the development work and architectural changes in PanDA to implement Global Shares, and describe how the Global Shares project has enabled the central control of resources and significantly reduced manual operations.

ATLAS Data Analysis using a Parallel Workflow on Distributed Cloud-based Services with GPUs

“…Driven by these motivations, the ATLAS experiment has pursued several initiatives to integrate cloud-based services with its existing distributed computing framework. Consequently, PanDA and Rucio have now been seamlessly integrated with the commercial cloud services [7], enabling ATLAS users to submit jobs to the Kubernetes clusters [8] offered by leading commercial cloud companies like Google and Amazon. As shown in Figure 4, the Harvester [9] module of PanDA interfaces with Kubernetes clusters to assign jobs efficiently to the available resources.…”

BigPanDA monitoring system evolution in the ATLAS Experiment

“…Initially, the project comprised a set of core views covering the basic needs of monitoring key aspects of the PanDA WMS, in particular jobs, tasks, files, datasets, computing sites, and users. With the advancement of the PanDA WMS [1] in the ATLAS Experiment, several new components have been developed, such as Harvester [10], iDDS [4], Data Carousel [5], and Global Shares [6]. All of them are closely related to key WMS objects and have to be monitored in an integrated way.…”

“…Since then, it has been continuously improved in line with the advancement of the PanDA WMS to meet new challenges brought by the ATLAS collaboration, in particular the integration of HPCs and commercial clouds, the central control of computing activity distributed among available resources, and orchestration of the data movement between tape storage and disks. To complete them, the several new components were developed, such as Harvester, Intelligent Data Delivery Service (iDDS) [4], Data Carousel [5], and Global Shares [6]. The overview of the ATLAS Workflow Management System is shown in Figure 1.…”

Managing the ATLAS Grid through Harvester

“…In order to follow the Global Shares [14] priorities of ATLAS, it is more desirable to unify all queues at a site into one single queue that can accept Pilots with different sizes. These unified queues can be managed using either the Pull mode or the new Pull UPS mode (see next subsection).…”