Madgraph5 GPU development

→ Europe/Zurich
Virtual (Zoom)

Virtual (Zoom)

# Madgraph dev meeting Monday 04.04.2022

Present: AV (notes), SR, OM, NN, WH, DS, SH, TC, CV

## Slides by Andrea and discussion

Slide 3.
OM: the difference between single-event and vector madevent mode is discussed in
https://github.com/oliviermattelaer/mg5amc_test/issues/4
The problem only appears in merged setups with MLM merging, eg ggtt+ggttg, not in ggtt or ggttg individually.
The difference is very small.
However it is important to fix this, CMS uses MLM merging routinely.
If we tell them not to use MLM, they would probably not use this version of MG5aMC.

Slide 3.
OM: agree that fixing https://github.com/madgraph5/madgraph4gpu/issues/341 can come later.
However it is important to fix it eventually.

Slide 4.
OM: prefer to keep the current madevent naming convention, eg P1_ll_ll.
This makes it possible for instance to reuse the same code for e and mu which are massless (tau is named differently).

Slide 4.
AV: did not mention that it works now because fortran has lowercase makefile and cudacpp uppercase Makefile.
Propose to rename the cudacpp Makefile as plugin.mk, for instance.
OM: this is a good idea, agree with that.

Slide 4.
OM: actually prefer to keep option 1, with the same directory P1_gg_ttx for fortran and cudacpp.
One can instead keep separate directories for source Fortran and cudacpp.
In case one can rename src as Source_cpp for instance.

Slide 4.
AV: if keep option 1, maybe will remove lib and keep the libraries all in P1? Maybe not.

Slide 4 and slide 5.
OM: for the multichannel strategy, note that there are two very different strategies in MadEvent.
The traditional SD strategy 1, and then a newer SD strategy 2.
OM will actually force the cudadcpp code to use the SD strategy 1, it does not make sense to use the SD strategy 2 in our case.
Note that get_channel_cut is only needed for SD strategy 2, hence we do not need to worry about that, indeed.

Slide 4.
OM: to test if the 30% difference is due to alphas, can try to use a fixed scale.
This is a parameter to set in the run cards.
Note that changing the run cards means that you need to rebuild the code:
this is because actually the build will create new code with those values hardcoded.

Slide 5.
SR: for alphas, just created a new PR, which includes also clang-format.
However, the alphas are only in the bridge and copied to memory, but they are NOT used yet.
SR is working on the second part of this PR.

Slide 5.
OM: note that CMS does the helicity picking a posteriori with an ad-hoc plugin for NLO.
This implies recalculating all matrix elements for the event, but performance-wise it is still
interesting because this only needs to be done for the accepted unweighted events,
not for every phase space point prior to unweighting.
This means that the random choice of helicity is low priority.
That said, implementing this in cudacpp is trivial, even simpler than the color choice.

Slide 5.
OM: Instead, the color choice is definitely something that we need to implement.
This may be a bit complex also because this is not just teh choice of color,
it is the choice of leadsing color (ie you must restrict the random choice to those colors that are leading colors).
Which colors are leading depends on the diagram.
In any case the algorithm exists in Fortran, so it can be copied to cudacpp.

Slide 2.
About the CMS talk: agreed that OM will give the talk,
and that it is no a problem if we are not fully ready with the code.
OM: the important thing is to warn them to be ready,
so that they can start trying out the code as soon as we give it to them.

## Round table

### OM

NTA

### TC

Finished kokkos port to golden tag X4.
Doing some performance tests with Nathan.

Q/TC: is madevent with cpp vectrization available?
OM: it is what AV has just discussed.
The vector size 16 is the number of events passed from fortran to cudacpp.
AV: this is not SIMD or GPU level, but it is the maximum level
(if you pass 16, you can only have 16 events at most).

TC/SR: it would be good to have a branch for golden tags
AV: agree, definitely we should have a branch

### SH

CI on virtual GPUs is running. It is using a Tesla T4.
Ran madgraph on it and it works, but did not make performance tests.
Please report any issues because would like to switch off the previous CI.

Q/AV: gcc11?
SH: no in the end gcc10 works, so using that.

### WH

NTR

### NN

Running some performance tests at Argonne, will update the twiki.

Q: do we plan to run at full device saturation?
AV: yes absolutely, now we are testing with 16, but we should aim for much more with GPUs!
OM/SR: agree, we should aim for full device saturation.

Also agree that it would be good to have a golden tag branch.

### DS

Confirm that three places in alpaka where 'static' must be added to constexpr.

Agree that it would be reasonable to have a golden tag branch.

SR: can you DS branch off on golden tag epochx4?
DS: yes will do that
AV: then you can cherry pick the addition of alpka directories there, and also static

Also investigating a few other performance differences, eg in the complex classes.

Confirm that for alpaka need device relocatable code.
This needd

Also experimenting with "never inline".
AV: three options in principle, keyword "always inline", no keyword, keyword "never inline".
AV: tried the comparison of the first two options.
DS: treied the third option, keyrowd "never inline". This is ebetter but only for one of the FFV functions,
performance degrades if you put it on all FFV functions...

### CV

Testing the build of alpaka.
Stil having issues, seems very dependent on package versions.

### SR

Put in a second pull request for alphas, including also clang format.
But it does not include the calculation yet (still working on it)

## AOB

Next meeting : 25 April (18 April is easter monday)

TC: submitted abstract to ICHEP.
Please check that we have all authors included.
(TC: will not go to Bologna though, someone else should, Nathan?)

TC: when are we having a nice collaboration meeting?!

There are minutes attached to this event. Show them.