System performance modelling WG meeting
Participants
- Local: Andrea Sciabà, Markus Schulz, Jan Iven, Johannes Elmsheuser
- Remote: Catherine Biscarat, David Lange, Gareth Roy, Renaud Vernet, Andrea Sartirana, Oxana Smirnova, Alessandra Forti, Graeme Stewart
Presentation on the ATLAS Memory Monitor
(refer to the slides)
Johannes: a network monitor has been added and is being valudated; it works by wrapping C network fucntions and dumps information to a log file. The overhead is not negligible, so we are still evaluating whether to run it at a large scale.
Jan: also xrootd can be easily instrumented.
Markus: it was shown that there can be a big difference even in a test environment between the amount of transferred data as measured by xrootd and at the network level.
Johannes: in ATLAS direct remote access is not used any more, FAX has been discontinued. We only read from local storage.
Markus: at the resource estimation meeting yesterday, a Tier-1 showed that half of their storage is remotely read by Tier-2s.
David: in CMS, compared to ATLAS, what changes is the balance between CPU and data access. AAA is routinely used by both analysis and production.
Johannes: there is R&D work in ATLAS on file caches, but nothing in production yet.
Workshop
Everybody who appears in the agenda as presenter or chair confirmed his/her availability. Markus proposes to meet in person some time on Monday or Tuesday to sort out the last details.
Task list
Workflow properties
Graeme: working on prmon (the Athena-agnostic version of Memory Monitor), will add network monitoring from /proc. OK if the application is generating all network traffic. Talked with David Smith, making advances.
Markus: we will have an intern to work on this (making measurements on workflows).
Gareth: updated the metrics Google doc with plots of metrics actually measured from real jobs.
Cost evaluation procedure
Catherine: now the power cost estimation if completely understood and the numbers agree with Renaud's ones, with the only difference being that he averages across the different generations of hardware actually installed.
Jan: I will update my calculations with the latest numbers from Bernd.
Catherine and Jan will discuss offline some details about the network.
At the workshop, Renaud will present a method for cost evaluation that can be used by other sites.
Toy model of workload
Markus looked at the current measurements of tracking time vs. pileup in ATLAS and a naive extrapolation to HL-LHC is scary. Graeme and Johannes stress that ATLAS is completely overhauling the reconstruction software (see talk from Andy Salzburger).
Markus is working on a parametric model for the tracking time (not to be intended for accurate predictions!).
Resource calculation
There was a meeting yesterday, mostly devoted to measurements of network traffic at some sites (CERN, PIC, GRIF).