HEPiX Benchmarking Working Group

Friday 24 Mar 2017, 13:55 → 15:00 Europe/Zurich

31/S-023 (CERN)

Manfred Alef (Karlsruhe Institute of Technology (KIT)), Domenico Giordano (CERN), Michele Michelotto

Meeting Date: 4/3/2017
Attendees:  D. Abdurachmanov, M. Alef,  L. Atzori,  B. Bockelman, J. Flix, A. Forti, D. Giordano, C. Grigoras, D. Lange, A. McNab, M. Michelotto,  A. Perez-Calero, M. Reis, A. Sciaba, L. Sexton-Kennedy

 

1) News (Domenico)

• Question about branch prediction effect on the python switch case of DB12: what would be the effect of a C++ implementation of DB12?
• Why DB12 is implemented in python? 
  • Andrew: because it came out from experimental evidence that it scales with the LHCb MC applications

2) CMS report (Pepe)

• Running ttbar MC events
  • known that there is an overhead of initializations and that also the last event takes long.
  • those are irreducible parts that motivate the choice of running on a large number of events 
• Large fluctuations of DB12 when running in the region of HT=ON
  • Comments: it's related to the adjustment of the CPU frequency. It would be useful to  fix the CPU frequency or at least monitor it
• Ratio plot: it would be better to normalise to the number of processes== number of physical cores, and not to 1, to better compare the performance of HS06, DB12 and ttbar in the HT region. Indeed it seems that HD06 and ttbar are correlated in that region.
• Andrea S. comment: CPU model is not available in CMS pilots
  • Answer: it will be added, work ongoing
• Action:
  • Pepe: to document how to run cmssw application and simulate ttbar events

14:00 → 14:10 News

Speakers: Domenico Giordano (CERN), Manfred Alef (Karlsruhe Institute of Technology (KIT)), Michele Michelotto

1)  Upcoming events and contributions from our WG:

• GDB meeting:  April 12th
  • Working group status update
  •  GDB is informed that the work is in progress, but the time scale is too short to give a definitive recommendation.
  • A longer discussion will take place during the WLCG Workshop (see next point)
  •  
• HEPiX Spring 2017 Workshop, 24-28 April 2017, Budapest (Hungary)
  • Summary report about the WG activity
  • in addition: a number of talks from other members of the WG (e.g. from A. Sciaba')
  •  
• 2017 WLCG Workshop (University of Manchester) 
  • Agenda still under preparation https://indico.cern.ch/event/609911/timetable/#20170619
  • The plan is to have a benchmarking session/discussion where to address the two subjects: 
    • fast benchmarks for pilots, clouds, etc
    • next generation of HepSPEC benchmarks
    •  

2) Next WG meeting (April 7th) will be dedicated to the review of the results for the GDB report

• Expect short report from all experiments’ representatives 
  • Statement about the correlation of DB12 and the experiment applications (sim, digi, reco)
  • Approach (in place or suggested) to enable reproducible studies of correlation among experiments’ workloads and next generation of benchmarks (HepSPECxx)
    • Document procedures, possibly share code (e.g. via CVMFS) of test applications

 

3) Documentation and Action list    

• A number of new sections have been introduced in the twiki
  1. Information needed to run passive benchmark  https://twiki.cern.ch/twiki/bin/view/HEPIX/CpuBenchmark#Passive_Benchmark
  2. Available resources for benchmarking activity (CERN, GridKa)
  3. DB12 Vs python version  
     • summarised already in email thread, now is in https://twiki.cern.ch/twiki/bin/view/HEPIX/DB12VsPythonVersion
     • need to reach a conclusion about this aspect: do we need to fix the python version to run DB12?
•  
• For the site representatives and the experiment representatives: Please remember to fill the information in our Twiki https://twiki.cern.ch/twiki/bin/view/HEPIX/CpuBenchmark
  • ​Thanks to who already did it    

4) AOB

• Extremely interesting talk of M. Guerri last Friday at the IT Technical Forum:
  • Title:  "Understanding applications’ performance”
    • aka understanding (i) DB12 magic boost and (ii) Intel Haswell core performance
  • The slides and the recording are available in indico https://indico.cern.ch/event/612774/

14:10 → 14:30 CMS DB12 tests

Speaker: Jose Flix Molina (Centro de Investigaciones Energéti cas Medioambientales y Tecno)

20170324_DB12s_JFlix.pdf