9–13 Jul 2018
Sofia, Bulgaria
Europe/Sofia timezone

GRACC: GRid ACcounting Collector

10 Jul 2018, 11:15
15m
Hall 7 (National Palace of Culture)

Hall 7

National Palace of Culture

presentation Track 3 – Distributed computing T3 - Distributed computing

Speaker

Brian Paul Bockelman (University of Nebraska Lincoln (US))

Description

The OSG has long maintained a central accounting system called Gratia. It uses small probes on each computing and storage resource in order to usage. The probes report to a central collector which stores the usage in a database. The database is then queried to generate reports. As the OSG aged, the size of the database grew very large. It became too large for the database technology to efficiently query to generate detailed reports.

The design of a replacement required data storage that could be queried efficiently to generate multi-year reports. Additionally, it requires flexibility to add new attributes to the collected data.

In this paper we will describe the GRACC architecture. GRACC uses modern web technologies that were designed for large data storage, query, and visualization. That includes the open source database Elasticsearch, message broker software RabbitMQ, and Grafana and Kibana as data visualization platforms. It uses multiple agents that perform operations on the data to transform it for easier querying and summarization.

Primary authors

Derek John Weitzel (University of Nebraska-Lincoln (US)) Marian Zvada (University of Nebraska Lincoln (US)) Brian Paul Bockelman (University of Nebraska Lincoln (US))

Co-authors

Kevin Michael Retzke (Fermi National Accelerator Lab. (US)) Shreyas Bhat (Fermi National Accelerator Laboratory)

Presentation materials