WLCG Information System Evolution Task Force

Europe/Zurich
513/R-068 (CERN)

513/R-068

CERN

1
Show room on map
Description
Meeting to discuss the evolution of the WLCG Information System

Attended:

Adrian, Alessandro Paolini, Balazs, Florido, George, Greg, Panos, Maarten, Alessandro Di Girolamo, Ian Neilson, Maarten, Andrew, Julia

 

First topic of discussion was the proposed format of the CE publishing json file

One block per queue + gatekeeper is foreseen

For cases when queue is not defined like  happens with condor, queue name will be omitted

Description should include the name of the site hosting computing capacity

Alessandro G. mentioned that the minimum set of requirements is that published info should be enough to generate JDL for job submission

The identical label in the first ans second block in the Balazs' proposal indicates that the these blocks represent a given set of resources and differ only by gatekeeper or queue. This means that this set of resources should be considered only once. For example, for accounting, if several CEs are publishing information for this set , only one number should be taken into account. This would allow to avoid double counting.

Need to include conversion factor time to work (hepspec) in every block

What about scaling factor? To be checked whether scaling factor is required to be exposed, or it is used internally by the batch system

Number of cores is missing. Might need more than one value for number of cores, may be max and min. This question caused some longish discussion. It was decided to take it offline. People were asked to come up with their proposals what would be the best option.

What to do with max running jobs, if the resource is shared by several VOs? Let's go for simple solution, no split per VO values, just overall for the time being

Same question is what to do with various parameters (max memory for example) which might depend on number of cores. Again start with simple solution always exposing values corresponding to single core. If we need more complexity, it can be added later.

Next steps

Balasz will create a new version in the googledoc, so that people can work on the document and add their comments.

People were asked to think about info to be published for number of cores

We would need to re-iterate through the proposal and then ask some pioneer sites to describe their resources in the proposed format (manually), publish URLs with the description in GocDB  attached to CEs . There is some URL attribute which exists and can be used for this purpose. Then experiments will be kindly asked to evaluate the file in terms of published info, whether  it is sufficient for their workflows.

Actually, no plan to include descriptor generation in coming ARC release as we initially planned. More thinking and evaluation is required.

 

George presented GocDB plans regarding new privacy requirements implementation.

Julia and Maarten commented that it is very useful example of concrete implementation for a concrete service. Can be considered for other WLCG central and experiment-specific services.

 

 

 

 

 

 

 

 

 

 

 

There are minutes attached to this event. Show them.