Choose timezone
Your profile timezone:
Powered by Indico
v3.3.3-pre
Participants
Local: Julia, Eddie,Costin, Tatiana, Miguel, Oliver, Alessandro, Nicolo, Alberto
Remote: Natalia, Ivan, Andrew, Stephan, Brian
Storage space accounting discussion:
Not all storage systems should provide all info with all protocols.
Tape also should be supported, how much is used, how much is available
We do not control all storage systems, we need an approach which would work also with Amazon, Google Compute, etc…
For storages which we do not control we can not assume that there is always a gateway provided on top of storage
Costin suggested that we require that if storage supports certain protocol it should also support space querying. Oliver thinks that storage providers might resist to this requirement. However, we might try to push for it.
Andrew argued in favour of a static document containing space info. This solves the "topology problem" (ie knowing which objects to query) and would be cacheable and thus queryable at high rate by clients.
There was a discussion regarding accounting vs monitoring. Miguel thinks that accounting information can not be used for operational needs. Agreed that monitoring can be used for accounting “downsampling” (while opposite not possible).
Natalia asked whether there is a plan to collect info centrally on the WLCG level and then experiments would query this central repository. For the time being this scenario is not foreseen. Experiments will continue to query data on their own, WLCG will do as well for the WLCG central accounting. No changes are foreseen in the experiments workflows at least for now.
There was a discussion regarding push/pull model. Pull model looks better, however some support for the push model can be also provided. To be seen later.
Tatiana suggested to look in the STAR structure for the structure of the usage record
What do we do with this information, who collects, aggregates, exposes it? There are two possible scenarios: this task is delegated to Apel. In this case we also need to have a detailed storage topology description most probably provided by CRIC and Apel has to be able to understand it and process it. Or we provide this functionality ourselves. We can decide on this later.
First we need to ensure basic things: 1). possibility to expose & access through CLI storage space info at the site level without SRM for all storage systems which are in use 2).describe storage topology on the WLCG level.
Summary: Alessandro and Oliver will work on a new version of the document taking into account all comments and will circulate the new draft.
OSG presentation
During the following discussion it was noted that the architecture of the new OSG accounting system is very similar to the architecture of the monitoring system at CERN. There is some difference regarding the way data aggregation is implemented.
What concerns storage accounting, we should work together with OSG in order to enable solution which would be interchangeable.