HEPiX Benchmarking Working Group

Friday 21 Apr 2017, 13:55 → 15:00 Europe/Zurich

31/S-023 (CERN)

Manfred Alef (Karlsruhe Institute of Technology (KIT)), Domenico Giordano (CERN), Michele Michelotto

Meeting Date: 21/4/2017
Attendees:  D. Abdurachmanov, M. Alef,  L. Atzori, S. Baymani, A. Brosa,  J. Flix, A. Forti, D. Giordano, D. Lange, A. McNab, M. Michelotto,  M. Reis, A. Sciaba

1) News (from Domenico) 

         - no comments

2) Containers in Atlas (from Lukas)

• Benchmarking is a use-case for container adoption
  • Provide reproducible, interactive development environments for personal development / software tutorials 
• Images can have different sizes
  • if cvmfs is already mounted on the host, the size of the image is rather small 2-3 GB
  • otherwise it can be as large as 10 GB, it includes all LCG libraries
  • sqlite condition file is another cause of large sizes.
    • D.G. comment: the condition data needed for the KV (particle gun) should be rather small (to be checked with the Atlas condition team)
  • image available lukasheinrich/athena_trfsbase 
• Checkpoints are under study to further speed up the execution, having snapshotted the initialisation step
  • Possibly ready for ACAT
  • Prototype available for tests
• Athena 21.x comes with open source licence

 

• Actions 
  • For Domenico: to try the Lukas' recipe to run KV with the usual athena version 17.x

3) Sandboxing CMSSW (from David)

• Attached documentation shows how docker images were created and used
  • indeed this work done in the context of OpenLab was eventually not much used
• David prefers to use directly tools like proot and care ( https://github.com/proot-me/PRoot )
  • see David documentation at  http://davidlt.web.cern.ch/davidlt/vault/proot/
  • sqlite file of conditions is big: 10 GB and is one of the contributors of the big size of the image. 
    • reason why adoption of squid caches is preferred.
    • NB: this implies a configuration at server level (as in the case of cvmfs) that not necessary is available for servers in internal networks in the commissioning phase.
    • To be checked with 

 

• Actions:
  • For David: to circulate a recipe to run a cmssw GEN-SIM workload (ttbar?), with the minimum amount of dependencies 
    • as a first step, still consider cvmfs as available.   

14:00 → 14:10 News

Speakers: Domenico Giordano (CERN), Manfred Alef (Karlsruhe Institute of Technology (KIT)), Michele Michelotto

Feedback from last GDB:

• Presentation of the WG activity at the last GDB was well perceived. 
  • https://indico.cern.ch/event/578985/
• Further discussion at the WLCG Workshop in Manchester 

Upcoming events and contributions from our WG:

• HEPiX Spring 2017 Workshop, 24-28 April 2017, Budapest (Hungary)
  • https://indico.cern.ch/event/595396/timetable
  • Contributions:

    • Updates from HEPiX Benchmarking Working Group; D. Giordano (via Vidyo, Wed 26/4 16:15 - 16:40)

    • Understanding the performance of benchmark applications; L.Atzori (Wed 26/4 17:30 - 17:55)

    • Understanding performance: optimisation activities in WLCG; A.Sciaba' et al. (Thu 27/4 12:05 - 12:30)

• WLCG Workshop 2017

  • 19-22 June 2017 University of Manchester

  • https://indico.cern.ch/event/609911/overview

 

14:10 → 14:30 How CPU benchmarking can profit from containers - The ATLAS use case

Speaker: Lukas Alexander Heinrich (New York University (US))

HEPIX_Talk.pdf

14:30 → 14:50 Sandboxing CMSSW for Isolated Execution (no internet connectivity)

Speaker: David Abdurachmanov (University of Nebraska-Lincoln (US))

CMSSW.openlab.re-execution.pdf