CPU Benchmark WG Meeting Notes, May 28, 2008


Present:
Alex Iribarren, Helge Meinhard, CERN/IT
Franco Brasolin, Alessandro de Salvo, CERN/ATLAS
Manfred Alef, FZK Karlsruhe
Peter Wegner, DESY Zeuthen
Michele Michelotto, INFN Padova
Martin Bly, RAL
Ian Gable, U Victoria


The top priority of this meeting was to discuss sources of error in both the experiments measurements and the SPEC measurements. Our goal is to understand our errors well enough to be able to do a proper chi square fit.

1. SPEC Sources of Error

• Alex reported variation in repeated measurement of both SPEC 2006 and SPEC 2000 of less then 1%. SPEC 2006 has slightly less variation then 2000.
• Alex is working to study the differences between the thread finish times.
• Manfred reported that he has run SPEC 2000 on 338 machines and reports that machines with identical hardware configurations have variation of less then one percent. Please see relevant HEPiX wiki page.

2. Experiment Sources of Error

• ATLAS has discovered that starting with the same random seed that simulation yields a different set of events for AMD and Intel. This will hugely impact the linearity given that only ten events are used for simulation. The reasons for it are unknown. Note that event generation did not suffer from this problem.
  
• Helge requested that each experiment verify that all their random seed sequences are the same between on all runs. Either check the seed after each event or look at the first and last seed.
• Helge suggested checking, as a test, if only the event Generation is linear in ATLAS since it does not suffer from these problems.

3. Action items

• Alex to continue study of threads for variability.
• Ian to contact Peter Hristov and Gabriele Benelli directly about checking random seeds.

4. Next meeting

• The next meeting will be held June 5, 15:00 UTC.