ROOT I/O Meeting

Friday 6 Sept 2019, 16:00 → 17:40 Europe/Zurich

32/1-A24 (CERN)

Brian Paul Bockelman (University of Nebraska-Lincoln (US)), Philippe Canal (Fermi National Accelerator Lab. (US))

Attendees: Philippe, Jakob, Guilherme, Jim, Pratyush Das.

Guilherme: Working on build system issues, in particular removing support for cmake -DTHREAD=OFF.

Philippe: I will take take of removing the code that does #ifndef _REENTRANT and add a check that it is defined.

Jim: Still working on Awkward C++ (long haul).  It will release through python PIP [install]

Jakob:  I mentioned a few time Andre Brickman  the head of the Storage Department at Mainz.   He will bring in a PhD student but not this year.  ATLAS and Belle2 jobs are run locally.  They are instrumenting the HPC system and asked us what they should measure.  We ought to give him an answer to help us and him (to tune the job).

Jakob: I visited Doug Thain’s group at Notre Dame.  His Cooperative Computing Tool Lab  has a nice set of product.  One that caught my attention is WorkQueue which helps in building master work application.  They are using Uproot.    They use in production to schedule CMS jobs.  We should definitively stay in touch with them in particular to help them understand ROOT/RNtuple files and to build up the Proof replacement.

Jakob: Work on progress ClusterCache (same goal as the TTreeCache) it (currently) can load cluster in full or partially or mmap.  It is using double buffering to load the ‘next’ cluster asynchronously.    I have discussed with Chris Jones about the interplay of ‘compute intensive I/O task’ with a multi-thread framework.  CMSSW use the “dot-then” abstraction.  I.e. not dependency per se but listing explicitly ‘do that when this task is finished”.

Jakob: We made progress on benchmark categorization.  This will help make sure that we cover as much of the (supported) phase space as possible.   Based on this discussion, I will produce a document to explicit the decomposition/categories we discussed.

Jakob: Parallel behavior of RNTuple.  RNTuple does not have any ‘global’ state; to have multiple thread-reading you ‘just’ create multiple copy of the RNTuple (this is used in RDF at the moment).   An ‘improvement’ could be to have the thread shared some of the cluster in memory; for this we would need some help from the user - a promise (explicitly or not) of which cluster a task/thread will cover.

Philippe: Next meeting in 2 weeks.

TODO: Give measure to Andre Brickman (starting next week).
TODO: Introduce Axel to Notre Dame’s WorkQueue.

 

16:00 → 16:20 Round Table