HEPscore Workshop
The goal of the workshop is to define the strategy for the adoption of HEPscore as replacement of HepSpec06.
The HEPSpec06 benchmark has been a reliable estimate of CPU performance for many years, and is currently used by the WLCG for accounting and pledges. However, HEPSpec06 is based on the SPEC2006 benchmark that is no longer supported. Further, it uses applications that do not reflect those used by the HEP community and will not provide benchmark for the new CPU+GPU systems.
The HEPscore Workshop will consist of several sessions:
Monday September 19
The first session will summarize the work of the WLCG HEPscore Task Force and the HEPiX Benchmark Working Group. During the session, potential candidates for the new HEPScore22 benchmark will be introduced.
The second session is devoted to the presentation of the current HEP-Workloads provided in the past year by the LHC experiments, Belle2, Juno and IWGN. Besides a description of the applications and of their performance, representatives of each experiment will highlight their expectations about the HEPscore22 composition and its lifetime.
The last session will introduce the framework developed to run HEP benchmarks and collect benchmark measurements ensuring traceability and monitoring. The HEP Benchmark Suite will be described and feedback about its usability will be provided by WLCG sites having used the suite.
Tuesday September 20
The morning session will focus on the policy and strategy foreseen to evolve from HepSpec06 to HEPscore, and the implications to accounting, pledging and procurement. The session will start with presentations from members of the Accounting Task Force, regarding the status of the development and the proposed deployment strategies. A round table with all the stakeholders will follow the presentations.
The last session is named "beyond x86". It will cover R&D work done by the Benchmarking Working Group in the area of heterogeneous computing, in order to extend HEPScore also to the benchmarking of servers with GPUs. In addition, there will be presentations on CPU power consumption and its relation to HEPScore.
Please register to the workshop to communicate if you will attend via zoom or in person.
Notes of the workshop are available for the registered participants as codiMD document in CERNbox at this url
-
-
HEPscore Task Force results
Review the activity of the Task Force and of the Working Group on HEPscore
-
1
Welcome and purpose of the workshopSpeakers: Domenico Giordano (CERN), Randy Sobie (University of Victoria (CA))
-
2
Overview of the Task Force activity
a recap of
- when and how it started
- who was part of it
- what we have achieved (the workloads, the campaign)
- where we have reported alreadySpeaker: Helge Meinhard (CERN) -
3
Analysis of the Measurement CampaignSpeakers: Randall Sobie (University of Victoria (CA)), Randy Sobie (University of Victoria (CA))
-
4
Considerations around HS06 and geometric meanSpeaker: Domenico Giordano (CERN)
-
5
HEPscore candidatesSpeaker: Tristan Sullivan (University of Victoria (CA))
-
6
DiscussionSpeaker: Round Table
-
10:40
Coffee break
-
1
-
HEP Workloads
Review the HEP workloads, their performance, the Experiments' perspective on SW evolution
-
7
Intro to the Workload sessionSpeaker: Domenico Giordano (CERN)
- 8
- 9
- 10
-
11
LHCb workloadsSpeaker: Andrea Valassi (CERN)
- 12
-
12:45
Lunch Break
-
13
Juno workloadsSpeaker: Xiaofei Yan (Chinese Academy of Sciences (CN))
-
14
IGWN workloadsSpeaker: Joshua Lee Willis
-
15
HEPscore development updateSpeaker: Christopher Henry Hollowell (Brookhaven National Laboratory (US))
-
7
-
HEP Benchmark Suite
Present the Benchmark Suite and gather feedback from site operators that have used it
-
16
Intro to the HEP-Benchmark-Suite sessionSpeaker: Domenico Giordano (CERN)
-
17
The Suite and the Benchmark DB (demo)Speaker: Gonzalo Menendez Borge (CERN)
-
18
Experience of running the Suite @ CERNSpeakers: Luca Atzori (CERN), Nikos Papakyprianou (CERN)
-
19
Experience of running the Suite @ RALSpeaker: Alastair Dewhurst (Science and Technology Facilities Council STFC (GB))
-
15:30
Coffee break
- 20
- 21
- 22
-
23
Experience of running the Suite @ IJCLABSpeaker: Dr Emmanouil Vamvakopoulos (Université Paris-Saclay (FR))
-
16
-
-
-
Strategy to deploy HEPscore
2 Items
- discuss the strategy foreseen by the accounting WG to deploy HEPscore beside HS06, and collect results
- recommend the HEP workloads mix and weight of HEPscore22-
24
Welcome and intro to the 2nd daySpeakers: Domenico Giordano (CERN), Randall Sobie (University of Victoria (CA))
-
25
HEPscore in the accounting system: Operational Strategy
- what is the deployment plan of a new benchmark?
- how sites should contribute?
- what is the possible time scale (months, years)
- how the pledges and procurement should then start to include HEPscore
(in parallel to HS06 at first, to then gradually replace it) - What are the validation procedures foreseen in these phases?
- how often a benchmark should/could be changed? every 4-5 years (at each LS?)
Speakers: Julia Andreeva (CERN), Maarten Litmaath (CERN) -
26
HEPscore in the accounting system: Development Status
- Description of the software changes, ETA, risks
- Implications /dependencies from other components that will require modifications (ARC and HTCondor CE, EGI portal, WAU, etc)
Speaker: Adrian Coveney (STFC UKRI) -
27
Considerations around a benchmark changeSpeaker: Bernd Panzer-Steindel (CERN)
-
28
Round table: HEPscore22 and adoption process
On the basis of what has been presented in the workshop, gather the feedback of the participants about:
- the HEP workload mix that better satisfies the WLCG requirements (representativeness, runtime duration, size)
- the operational process for HEPscore adoption in the Accounting system (timescale, validation, frequency of future upgrades, ...)Speaker: round table -
10:35
Coffee break
-
24
-
Developments beyond x86 CPUs
We will cover the extension of HEPscore to GPUs workloads, the build of CPU workloads on ARM/Power, fast benchmarks, the usage of HEP workloads for measurements of Power consumption
-
29
Intro to the sessionSpeaker: Domenico Giordano (CERN)
-
30
Experience running HEP workloads at HPCSpeaker: David Southwick (CERN)
-
31
Madgraph on GPUSpeakers: Andrea Valassi (CERN), Stefan Roiser (CERN)
-
32
Study of ML Particle Flow based GPU benchmarkingSpeakers: David Southwick (CERN), Mr Eduard Cuba (University of Zurich (CH)), Eric Wulff (CERN)
-
33
CMS HLT-likeSpeakers: Dr Andrea Sciabà (CERN), Dr Tommaso Boccali (INFN Sezione di Pisa), Tommaso Boccali (INFN Sezione di Pisa, Universita' e Scuola Normale Superiore, P)
-
12:30
Lunch Break
-
34
Study of power efficiency ARM vs x86Speaker: Emanuele Simili (University of Glasgow (GB))
-
35
Measurements of electrical power consumption for HEP workflows @ CERNSpeakers: Luca Atzori (CERN), Nikos Papakyprianou (CERN), Stefan Roiser (CERN)
-
36
Measurements of Power Consuption (room for other contributions)Speaker: round table
-
29
-
Strategy to deploy HEPscore: Conclusions
2 Items
- discuss the strategy foreseen by the accounting WG to deploy HEPscore beside HS06, and collect results
- recommend the HEP workloads mix and weight of HEPscore22
-