Present: Danilo, Gerri, Enric, Guilherme, Pere, Enrico, Jim New Actions: Open Actions: * Lorenzo: integrate PR2 of Xavi (#367) about vectorisation * Lorenzo, Danilo: merge VecCore PR once discussion about it is complete - Now the comments are all integrated - Need to discuss further involving Pere if needed * Danilo: integrate new ctors taking a TEntryList in the TDF - Make sure to throw an informative message if more than one tree is present in the file(s) --> This is pending until the creation of TEntryLists is addressed with an Action. * Gerri, Enric, Xavi, Danilo: identify a chunking (packetising) procedure for MT, MP, collections and trees (perhaps respecting clusters' granularity a la TTreeProcessor) * Xavi: Slide 7 2N debug nasty scaling obtained. Ideas: - With perf count the page faults (Pere's idea). - Artificially increase the work to see if the overhead is less. - Try only one worker. * Xavi: try fma with Ofast before ruling that out
Closed Actions: * Enric, Danilo: assess the difficulty of enabling TTreeProcessorMT also in the non imt builds, absorbing the duality of the imt/non-imt code path at the level of the ttreeprocessor implementation rather than TDF. - Now TTreeProcessorMT depends on TThreadExecutor and it makes little sense to promote it out of IMT builds * Enric, Xavi: investigate the reimplementation of TTreeProcessExecutorMT in terms of TThreadExecutor - Done. * Xavi, Danilo, Gerri: how to input the parameter to create the size of the partitions in MapReduce - Done: we opt for having the possibility to input the number of chunks as Spark allows to specify the number of partitions * Xavi, Danilo: look into the implementation of the TTreeeProcessorMT::Process:Done, plus GetEntry and FlushBasket News: * A clear roadmap to exploit jitting in order to avoid to specify types when dealing with TDF transformations and actions has been identified - First steps done in the understanding of how filters can be expressed as C++ strings - These would by construction address many (all?) issues relative to the TDF-PyROOT integration * A clear roadmap has been identified for implementing ranges as "filters able to communicate their completion status to the tdf node" Jim: PyData has a list of python compilers which could help us in translating python to C++. Pythran was mentioned: http://compilers.pydata.org/