HEPiX Benchmarking Working Group

Friday 15 Feb 2019, 13:55 → 15:00 Europe/Zurich

31/S-023 (CERN)

Manfred Alef (Karlsruhe Institute of Technology (KIT)), Domenico Giordano (CERN), Michele Michelotto

Present: Michele Michelotto, Domenico Giordano, Manfred Alef,  JeanMichel Barbet, Chris Hollowell, Andrea Sciaba, Martin Bly, Luca Atzori,  Lorenzo Rinaldi, Aritz Brosa

Notes taken by Michele

Domenico presents the status of the Reference Workload Package that has been recently presented to the last GDB (Jan 13th). There are now two separate gitlab project that Domenico will merge in a unique project after some big work from Luca A. 

There is also a JIRA project to keep track of the progress. It is a tool commonly used by sw developer but not familiar to everybody so Domenico explain the tools and how the project has been organized in the tool, e.g. the sub task for the experiments, the infrastructure and other business. These subtask are called EPICS in Jira. 

From the next meeting we will start going through all the ticket and activities to check the completeness, the priority and the progress. In particular the INFRA has made a lot of progress lately. 

Lorenzo R. propose to put all the Open Data on CVMFS

Then there is a report from Manfred Alef on the reproducibility of the HEP workload benchmarks. Very good for LHCB, not good (20%) for Atlas and very bad for Alice (x50). There is a huge fraction of job failing inside Geant3 and the source of the failure is not understood. However Manfred points out that the Benchmark should, and at this moments can't distinguish, a good run from a bad run and this can alter the score. 

Looks also at the minutes prepared by the two authors in advance. 

 

 

14:00 → 14:20 Reference Workload Packages

Speaker: Domenico Giordano (CERN)

Jira

Twiki

Presentation given at the last GDB 

Plan: 

• Migrate hep-workloads and cloud-benchmark-suite repos
• Start Jira sprints (from next meeting)

14:20 → 14:35 Reproducibility of HEP workload benchmark results

Speaker: Manfred Alef (Karlsruhe Institute of Technology (KIT))

• Repeated runs of the containerized HEP workload packages on 3 generations of test systems (Intel Sandy Bridge/Haswell/Broadwell) forking either only 1 single benchmark copy, 1 copy per core, ~1.5 copies per core, or 2 copies per core (1 copy per logical processor)
  • https://twiki.cern.ch/twiki/bin/view/HEPIX/HEPWorkloadReproducibility
• Workload images:
  • alice-gen-sim
  • atlas-gen-bmk
  • lhcb-gen-sim
  • cms-ttbar-sim not tested because of Frontier issue
• Preliminary (!) results:
  • LHCb: low spread ±2%
  • ATLAS: fluctuation range of ~20%
  • ALICE: huge spread (factor of ~50)
• Fluctuations in the ALICE results are caused by a bug in Geant3. Some tasks crash immediately, nevertheless the containerized ALICE workload prints a final performance score instead of an error message. The more copies are running in parallel the more tasks will crash.
  • When launching 40 benchmark copies on a system under test with 40 logical processors then almost all runs have at least one crashing task; average: ~22 successfull tasks!
• Benchmark tool should parse logfiles for error messages, and print a final performance score only for successfull runs
  • Experiment experts, please provide appropriate diagnostic tools!
  • Thanks to Costin Grigoras for checking the logfiles of the ALICE runs!