5th EGEE User Forum

The APEL CPU Usage Accounting Infrastructure for NGI Grids

Apr 12, 2010, 5:27 PM

3m

Aula (Uppsala University)

Poster Support services and tools for user communities Poster session

Speakers

Mrs Cristina Del Cano Novales (e-Science Centre, Science and Technology Facilities Council, United Kingdom)Mr Gilles Mathieu (e-Science Centre, Science and Technology Facilities Council, United Kingdom)Mr John Casson (e-Science Centre, Science and Technology Facilities Council, United Kingdom)Dr John Gordon (e-Science Centre, Science and Technology Facilities Council, United Kingdom)Dr Ming Jiang (e-Science Centre, Science and Technology Facilities Council, United Kingdom)Mr William Rogers (e-Science Centre, Science and Technology Facilities Council, United Kingdom)

Description

In order to implement and deploy a scalable and flexible distributed CPU usage accounting infrastructure for the NGI Grids, the accounting records transport mechanism of the APEL (Accounting Processor for Event Logs) tool is modified and extended to integrate with ActiveMQ message broker network. The new APEL CPU usage accounting infrastructure supports a robust accounting capability at an NGI level and flexible across VOs accounting records queries.

Impact

Within the distributed accounting infrastructure, accounting records are transported from APEL Publishers at Grid sites to either a regionalised accounting system or the central one by choice via a common ActiveMQ message broker network. This provides an open transport layer for other accounting systems to publish relevant accounting data to a central accounting repository via a unified interface provided an APEL Publisher and also will give regional/NGI Grids the flexibility in their choice of accounting system. The robust and secure delivery of accounting record messages at an NGI level and between NGI accounting instances and the central one are achieved by using configurable APEL Publishers and an ActiveMQ message broker network.

Detailed analysis

A general topic publication and subscription messaging model enables distributed components in a system to publish and subscribe messages to/from a well defined topic that can be viewed as a virtual destination and source of messages. The definition of topics and low level reliable delivery of messages among components can be achieved by using a concrete message broker implementation to this model. The work reported here investigates the feasibility of adopting such a messaging model to implement a distributed accounting infrastructure and utilises the Apache ActiveMQ message broker network as the accounting records transport layer of APEL for robust delivery of accounting record messages. In the distributed infrastructure, while the ActiveMQ message brokers will manage the delivery of accounting records messages, at a NGI level and between NGI accounting instances and the central records cache, the original user interfaces for existing APEL clients will remain consistent.

Conclusions and Future Work

The implementation and testing with production CPU usage records publication of the new APEL based accounting infrastructure demonstrate that the transport mechanism of a distributed accounting infrastructure can be implemented based on topic publication and subscription messaging model in an ActiveMQ message broker network. Further investigations on the scalability and fault tolerance features of ActiveMQ broker will be conducted.