GRACC: New generation of the OSG accounting

Abstract

Abstract. Throughout the last decade the Open Science Grid (OSG) has been fielding requests from user communities, resource owners, and funding agencies to provide information about utilization of OSG resources. Requested data include traditional accounting -core-hours utilized -as well as users certificate Distinguished Name, their affiliations, and field of science. The OSG accounting service, Gratia, developed in 2006, is able to provide this information and much more. However, with the rapid expansion and transformation of the OSG resources and access to them, we are faced with several challenges in adapting and maintaining the current accounting service. The newest changes include, but are not limited to, acceptance of users from numerous university campuses, whose jobs are flocking to OSG resources, expansion into new types of resources (public and private clouds, allocation-based HPC resources, and GPU farms), migration to pilot-based systems, and migration to multicore environments. In order to have a scalable, sustainable and expandable accounting service for the next few years, we are embarking on the development of the next-generation OSG accounting service, GRACC, that will be based on open-source technology and will be compatible with the existing system. It will consist of swappable, independent components, such as Logstash, Elasticsearch, Grafana, and RabbitMQ, that communicate through a data exchange. GRACC will continue to interface EGI and XSEDE accounting services and provide information in accordance with existing agreements. We will present the current architecture and working prototype.

Testing the limits of HTTPS single point third party copy transfer over the WAN

“…As a next step, we started deploying HTTP via XRootD at four US CMS Tier 2 centers for production transfers to gain long term operational experience. Production transfers via this infrastructure are being tracked using GRACC [12]. Initial experience with this production deployment is shown in Figure 5.…”

Cosmos : A Unified Accounting System both for the HTCondor and Slurm Clusters at IHEP

“…Many systems [1][2][3][4][5] are developed to account resource usage. These systems have similar functional components but different implementations.…”

Factory Monitoring for the 21st Century

“…However, further work had to be done to take the new monitoring a step further and provide a way for data to be pushed to more than one backend. The Grid Accounting Collector (GRACC) [7] is the official monitoring Elasticsearch instance for OSG. It is a natural place to also back up all factory statistics.…”