Choose timezone
Your profile timezone:
Powered by Indico
v3.3.3-pre
Attended: Dimitrios, Oliver, Salvatore, Costin, Eddie, Julia, Alsaandro, Ivan Dias, Ivan Kadochnikov, John, Pepe, Adrian, Alessandra, Natalia
Comments/discussion during Julia’s presentation
Julia asked Costin about storage space accounting in ALICE. Costine told that it is pretty complete and information is even probably pushed to the WLCG monitoring infrastructure, so can be used in the accounting prototype.
John mentioned work on storage space accounting of EMI/EGI. Julia told that she was aware about this effort and she was planning to invite EGI colleagues to the next meeting and ask them to present the status.
Oliver mentioned that even not all attributes which are mentioned in the presentation for the use case of the global view are not mandatory. Essentially what is mandatory is to know how much space is used and free for a particular area. Other attributes are nice to have. So in the report we should foresee some fileds which are mandatory, others are optional. Validation for various attributes can be different.
There was a clarification from Alessandro regarding sort of “dynamic” use case. Experiments would like to estimate how much free space they have while organization their processing.
Comments/discussion during Dimitrios’ presentation
There was a discussion about possible modes of data publishing from the primary source. There are several possible options:
Push mode: Currently used by Apel. Apel clients are pushing information into message queue. According to John, it is implemented for dpm and dcache and for ~25 sites. One can view data in the accounting portal:
http://accounting-devel.egi.eu/storage.php?query=logical_tb_used_avg&startYear=2017&startMonth=1&endYear=2017&endMonth=4&yrange=SITE&xrange=DATE&groupVO=lhc&chart=GRBAR&scale=LIN&localJobs=onlygridjobs
There are still some bugs to be fixed though. It might be not straight forward to instrument all storage implementations for this pushing mode.
Pull mode:
Two options
a). The collector queries CRIC for storage space areas topology and then loop over all areas querying the storage for every particular area
b). The storage is publishing information in one go in structured way (json), for example on some URL from where collector retrieves this data
According to Oliver pull mode option b would be the easiest one from the point of view of storage providers and might be the one to chose if we want to progress quickly.
AOB discussion
Alessandro raised an issue regarding raw wallclock publishing in APEL.
John told that for sites which did publish start and end time stamps, raw wallclock could be calculated out of these time stamps. This covers ~75% of sites. For other 25 which do not publish start and end time stamps for every job and/or publish rather summary records, in order to add raw wallclock metric one would need to change an Apel client. This brought us to the discussion of understanding which other possible attributes WLCG would like to have which would require changes in the Apel client. One of the requirements would be a possibility to publish multiple benchmarks. Currently only one is possible. Additional complication is that HEPSPEC06 benchmark is currently retrieved from BDII. If we have other benchmarks which are run inside pilots, how they are communicated to APEL client? MJF?
In summary we concluded that we need to collect all WLCG requirements which would imply changes in the Apel client and DB schema and will discuss them with the Apel developers at the next meeting.