Distributed Monitoring Infrastructure for Worldwide LHC Computing Grid

Abstract

The journey of a monitoring probe from its development phase to the moment its execution result is presented in an availability report is a complex process. It goes through multiple phases such as development, testing, integration, release, deployment, execution, data aggregation, computation, and reporting. Further, it involves people with different roles (developers, site managers, VO managers, service managers, management), from different middleware providers (ARC, dCache, gLite, UNICORE and VDT), consortiums (WLCG, EMI, EGI, OSG), and operational teams (GOC, OMB, OTAG, CSIRT). The seamless harmonization of these distributed actors is in daily use for monitoring of the WLCG infrastructure. In this paper we describe the monitoring of the WLCG infrastructure from the operational perspective. We explain the complexity of the journey of a monitoring probe from its execution on a grid node to the visualization on the MyWLCG portal where it is exposed to other clients. This monitoring workflow profits from the interoperability established between the SAM and RSV frameworks. We show how these two distributed structures are capable of uniting technologies and hiding the complexity around them, making them easy to be used by the community. Finally, the different supported deployment strategies, tailored not only for monitoring the entire infrastructure but also for monitoring sites and virtual organizations, are presented and the associated operational benefits highlighted.

DIRAC universal pilots

“…We create a new pilot command for running SAM [11] (Service Availability Monitoring) tests at sites, and report the outcomes required by WLCG. This approach allows SAM tests to be run on resources (such as VMs created spontaneously in the Vacuum model) that have been previously inaccessible to the WLCG site testing framework.…”

Service Availability Monitoring Framework Based On Commodity Software

“…The continuous monitoring of the WLCG infrastructure is a complex task and a large number of monitoring tools are involved in the exercise [1]. At the core of this activity is the Service Availability Monitoring (SAM) framework [2], which hides this complexity and orchestrates the monitoring workow execution, from the implementation of probes to the presentation of metric results. In addition, SAM brings to the WLCG management, VOs [3], and sites, a powerful and accurate view over the WLCG resources.…”