WLCG Accounting Task Force Meeting

Europe/Zurich
513/R-068 (CERN)

513/R-068

CERN

19
Show room on map
Description

Zoom link:

https://cern.zoom.us/j/63355885303

Password is sent to the mailing list

 

Attended:

Alessandro Paolini, Adrian Coveney, Dominico Giordano, Helge Meinhard, Maarten Litmaath, Dimitrios Christidis, Renato Sanatana, Gonzalo Menedez Borge, Jaroslava Schovancova, Julia Andreeva

Discussion after Domenico's presentation

Versioning of the bechmark. Important at the current stage since allows to compare results with various benchmark versions. Useful to be able to keep the track of the benchmark version in the accounting workflow. However, in a long term, when the new benchmark is deployed in production, we do not foresee often version change. The hope is to have a stable benchmark which can be used for at least for 5-6 years.

The new benchmark should be a representative set of various workflows, not specific for a given experiment or for LHC in general, capable to well measure power of the CPU resource of the EGI and WLCG.

Current focus is on the CPU benchmark, not targeted for GPU yet, however it was already tried for ARM and might be relevant for ARM as well.

Current benchmark suit should also work for GPU under the condition that experiment workloads are capable to exploit GPU resources. There is a hope that when experiments are capable to use GPU resources , we can re-use already developed machinery.

 

Discussion after Adrian's presentation

How we foresee to do a switch.

Domenico described the situation when the site has a cluster with old hardware which is benchmarked with HS06 and then purchases new hardware when a new benchmark is ready for production. Whether in this case two groups of the site resources would be benchamrked with two different benchmarks?

Most probably, when the new benchmark is ready for deployment we will require all resources to be re-benchmarked with HEPSCORE, not willing to have a mixture and to drag HS06 any longer. We know that already now many sites do not have proper assessment of their resources with HS06. So this would be an opportunity to benchmark all site CPU resources properly across our infrastructure.

Discussion after Alessandro's presentation

Maarten asked why APEL client needs to use site BDII for getting benchmarking factor. Doesn't it introduce unneeded dependency?

Benchmarking factors for the computing clusters are often required for operations (experiment and central ones) to understand accounting data. That is why it is handy to have it available via BDII query or via CRIC API (in the WLCG context). Since EGI continues to rely on BDII and therefore majority of the  EGI sites will run site BDII instances, this is not a big overhead to provide bechmarking factors via BDII. Alessandro showed that it would require minor change. APEL client is able to get benchmarking factor from either source, the only thing we should be careful about, that in case benchmarking factor is published both in BDII and local configuration, it is consistent in both sources.

For ARC CE and HTCondor CEs  in case they change normalized data to APEL, some changes on the CE side will be required. We need to coordinate with ARC and HTCondor development teams.

EGI portal changes

Ivan could not attend the meeting. During the preliminary discussion with Julia and Adrian he confirmed that changes required in the portal (in order to show CPU consumption with HEPSCORE in parallel with HS06) are not big and he will take care about these changes.

Changes in WAU

Julia mentioned, that in order to preserve the current layout of the EGI accounting portal and minimize changes required in the portal to support two benchmarks in parallel, the possibility to compare consumption with both benchmarks will be provided in WAU. There is a similar functionality implemented in WAU  to compare generated and validated CPU and storage consumption.

 

Next steps

- Alessandro will get in touch with ARC and HTCondor developers to discuss changes required on their side to support two benchmarks in parallel

- We will ask for 2 slots  at the GDB in February, to present progress regarding both areas of work: new benchmark  and enabling possibility to have two benchmarks in the accounting workflow during the transition period

- We might schedule a next meeting to see how we progress sometime early next year

 

 

 

 

 

 

There are minutes attached to this event. Show them.