ROOT Parallelisation, Performance and Programming Model Meeting

42-R-406 (CERN)



Show room on map
Danilo Piparo (CERN)
CERN number: 71400 Extension: 109284483#
Videoconference Rooms
ROOT Team Meeting
Fons Rademakers
Auto-join URL
Useful links
Phone numbers
Present: Danilo, Guillerme, Xavi, Enrico, Enric, Lorenzo, Philippe, Kim, Gerrim, Jim

New Actions
* Guillerme: report on the progress about the veccore integration and coherency with vc and vecgeom
* Danilo: integrate new ctors in the TDF
   - Make sure to throw an informative message if more than one tree is present in the file(s)
* Danilo: investigate the TProfiles in TDF, investigate usage of histos with buffers
* Danilo: PR #364, integrate
* Xavi, Danilo: iron out the issue streaming the TSeq between processes
* Gerri, Enric, Xavi, Danilo: identify a chunking (packetising) procedure for MT, MP, collections and trees (perhaps respecting clusters' granularity a la TTreeProcessor)
* Xavi: implement Kahan summation to debug the different results in the likelihood benchmark based on multiprocessing
* Enric, Danilo: assess the difficulty of enabling TTreeProcessorMT also in the non imt builds, absorbing the duality of the imt/non-imt code path at the level of the ttreeprocessor implementation rather than TDF.
* Xavi: Slide 7 2N debug nasty scaling obtained. Ideas:
   - With perf count the page faults (Pere's idea).
   - Artificially increase the work to see if the overhead is less.
   - Try only one worker.
* Xavi: deliver PR1 for vectorisation in math to be integrated in ROOT as well as the associated tests

Open Actions

Closed Actions
* Guilherme: if possible, report on the status of the integration of veccore
   - Created an independent version, working to integrate it in vecgeom and ROOT. Start from vecgeom because smaller and quicker during the compilation
   - Challenge: Get vecgeom, vc, veccore coherently - needed a bit more time to make it work
   - Issues with the VC integration and Ninja: working on those
* Xavi: communicate to Guilherme the issue he had with vecore
* Danilo, Enrico, Enric: decide an interface for trees on multiple files (or chains)
* Danilo: make progress on the 2D and 3D histos
* Danilo: merge PR 337 and tests
* Action Xavi: check the table at slide 12 pres 2, redo numbers only with 4 physical cores.
* Action Xavi: chunk the work in N parts with N being the number of processes.
* Action Xavi/Danilo: Work out the difficulty streaming the double_v.

Xavi's Talk
o ProcessExecutor: creating a lot of workers with many points. Implement the interface for chunking which was there in the TThreadExecutor into the TProcessExecutor (#364)
o Solved with the ifdef techinque the streaming of veccore types
o Problem: issues in the likelihood calculations, idea of Guillerme: try out a different backend. But in the mt case veccore works.

Danilo's Talk
o TDataFrame: Philippe proposes that if we have more than one tree, we should throw rather than pick the "first" one.
o Kim: the exception message should be informartive
o Gerri: we accept the challenge of inglobating the multiprocessing as well as distributed system
o Lambda Macro:
  - we should check if this solution suffers from "comma related" problems
  - other potential problem: using quotes

Guillerme asked gsoc about packaging HEP software within the Gentoo community
There are minutes attached to this event. Show them.
    • 16:00 16:10
      Updates on multiprocessing and fitting parallelisation 10m
      Speaker: Xavier Valls Pla (University Jaume I (ES))
    • 16:10 16:55
      Round table 45m
      • News
      • Actions from last week
      • New TDataFrame constructors
      • An alternative way to express C++ lambda functions
      • Round table