12-16 April 2010
Uppsala University
Europe/Stockholm timezone

Application Domain Accounting for EGI

Apr 12, 2010, 3:45 PM
15m
Room IX (Uppsala University)

Room IX

Uppsala University

Oral Software services exploiting and/or extending grid middleware (gLite, ARC, UNICORE etc) Infrastructure Tools and Services

Speaker

Dr Javier Lopez (CESGA)

Description

Accounting is a powerful tool for users and VOs to obtain information on grid resources usage. Currently, they can access the accounting portal to display executed jobs. However, grid users and VOs prompt further improvements to enhance the EGEE accounting system and foster cooperative endeavors. One of the most requested features is application-level accounting. Together with COMPCHEM VO the initial requirements are being analyzed. It has suggested to gather extra information on the most executed programs, on the amount of retrieved results (per user/program) and on the used resources.

Impact

The principal aim of this proposal is to further improve the EGEE accounting system. If these improvements are implemented in next years, they will have an immediate benefit for grid users and SSCs. One of the most visible improvements will be reflected in the EGEE accounting portal, based on the current internal application used at CESGA since 2003, a new method to gather and export the application usage will be used. This information will be aggregated, filtered and published for each site and later it be integrated in the accounting portal. Several new reports will be produced going from which application consumes more CPU time to which scientific area is using more computational resources. Other reports can be used to assist administrators, VO managers and users discovering application performance loss or application failures. Additionally, it is possible to detect in advance problems with the applications (because exit codes can be recorded) and which applications deserve more attention by the managers of the site, as well as, bad usage of the resources. One of our goals is to show and discuss this proposal to other SSCs to contribute accounting enhancement.

Conclusions and Future Work

Application-level accounting was originally requested by COMPCHEM VO to improve its own portal, but this work can be of benefit for most of the EGI SSCs. It would be useful for users and VOs to establish in EGI a common science gateway able to record some information that is not collected by the current grid middleware. This is not an easy task, and involves the development of new sensors, the agreement between different EGI groups like ROSCOE and EMI, and the installation of these new sensors on grid sites. This should be discussed and developed over the next fewyears.

Detailed analysis

The current EGEE III accounting system is oriented to gather job-level information from the batch system and merge it with VO and user-level information. This kind of information, however, is not detailed enough although there is demand on filling such a gap with application-level accounting. At present VOs have no information whatsoever on the software utilized and on the results gathered by the users. The development of application-level accounting tools will require the joint effort of different development teams along the next years. COMPCHEM and CESGA are beginning to design a preliminary draft which contains specific requirements about users and services parameters. To satisfy this new application-level accounting request, gLite middleware will require some improvements, to collect and store the new information in the EGI accounting repository database. On the other hand the new fields to be stored should be agreed and validated by Open Grid Forum (OGF) to become part of the UR standard. UF will be a good opportunity to present the work done allowing other VOs and SSCs to comment about it and add their requirements.

Keywords Application domain accounting,application-level resource usage,VOs and SSCs management optimization
URL for further information http://www.egee.cesga.es/Accounting

Primary authors

Mr Alvaro Simon (CESGA) Andres Gomez (Unknown) Mr Antonio Laganà (University of Perugia) Aurelio Rodriguez (CESGA) Mr Carlo Manuali (University of Perugia) Dr Carlos Fernandez (CESGA) Mr Esteban Freire (CESGA) Dr Javier Lopez (CESGA) Mr Ruben Diez (CESGA) Mr Sergio Diaz (CESGA)

Presentation materials