ROOT Parallelisation, Performance and Programming Model Meeting

Thursday 9 Mar 2017, 16:00 → 18:25 Europe/Zurich

42/R-022 (CERN)

Danilo Piparo (CERN)

CERN number: 71400 Extension: 109284483#

Present: Danilo, Gerri, Enric, Guilherme, Pere, Enrico, Jim

New Actions:

Open Actions:
* Lorenzo: integrate PR2 of Xavi (#367) about vectorisation
* Lorenzo, Danilo: merge VecCore PR once discussion about it is complete
  - Now the comments are all integrated
  - Need to discuss further involving Pere if needed
* Danilo: integrate new ctors taking a TEntryList in the TDF
   - Make sure to throw an informative message if more than one tree is present in the file(s)
   --> This is pending until the creation of TEntryLists is addressed with an Action.
* Gerri, Enric, Xavi, Danilo: identify a chunking (packetising) procedure for MT, MP, collections and trees (perhaps respecting clusters' granularity a la TTreeProcessor)
* Xavi: Slide 7 2N debug nasty scaling obtained. Ideas:
   - With perf count the page faults (Pere's idea).
   - Artificially increase the work to see if the overhead is less.
   - Try only one worker.
* Xavi: try fma with Ofast before ruling that out

Closed Actions:
* Enric, Danilo: assess the difficulty of enabling TTreeProcessorMT also in the non imt builds, absorbing the duality of the imt/non-imt code path at the level of the ttreeprocessor implementation rather than TDF.
 - Now TTreeProcessorMT depends on TThreadExecutor and it makes little sense to promote it out of IMT builds
* Enric, Xavi: investigate the reimplementation of TTreeProcessExecutorMT in terms of TThreadExecutor
 - Done.
* Xavi, Danilo, Gerri: how to input the parameter to create the size of the partitions in MapReduce
 - Done: we opt for having the possibility to input the number of chunks as Spark allows to specify the number of partitions
* Xavi, Danilo: look into the implementation of the TTreeeProcessorMT::Process:Done, plus GetEntry and FlushBasket 


News:
* A clear roadmap to exploit jitting in order to avoid to specify types when dealing with TDF transformations and actions has been identified
  - First steps done in the understanding of how filters can be expressed as C++ strings
  - These would by construction address many (all?) issues relative to the TDF-PyROOT integration
* A clear roadmap has been identified for implementing ranges as "filters able to communicate their completion status to the tdf node"


Jim: PyData has a list of python compilers which could help us in translating python to C++. Pythran was mentioned: http://compilers.pydata.org/

 

16:00 → 16:45 Round table