Alice Weekly Meeting: Software for Hardware Accelerators
→
Europe/Zurich
-
-
10:00
→
10:20
Discussion 20mSpeaker: David Rohr (CERN)
Color code: (critical, news from this week: blue, news from last week: purple, no news: black)
Sync reconstruction
Async reconstruction
- Need to investigate short GPU stall problem.
- Limiting factor for pp workflow is now the TPC time series, which is to slow and creates backpressure (costs ~20% performance on EPNs). Enabled multi-threading as recommended by Matthias - need to check if it works.
- Will tune existing 16-core settings, add a SITEARCH for 16core CPU, and 16coreCPU + generic NVIDIA / AMD GPU, like for 8 core.
- Will retune EPN async workflow for TPC + ITS on GPU on 2025 data.
GPU ROCm / compiler topics:
- Problem with building ONNXRuntime with MigraphX support.
- Need to find a way to build ONNXRuntime with support for CUDA and for ROCm.
- Try to find a better solution for the problem with __device__ inline functions leaking symbols in the host code.
- Bumped to CUDA 13.4 and ROCm 10.1, we can now bump to GCC 16.2.1
- SLC10-FullCI running and looks healthy. We should now drop the SLC9-FullCI, and move the remaining required tests from in there to the normal SLC9-CI.
TPC / GPU Processing
- WIP: Use alignas() or find a better solution to fix alignment of monte carlo labels: https://its.cern.ch/jira/browse/O2-5314
- Need to check the problem with ONNX external memory allocator.
- Next high priority topic: Improvements for cluster sharing and cluster attachment at lower TPC pad rows. PR: https://github.com/AliceO2Group/AliceO2/pull/14542
- Check for unnecessary f64 instructions in GPU code.
- Significant progress in TPC tracking developments. Phase 2 changes are completed, need few more days of validation and tuning, and can be merged. The extrapolation feature of Phase 3 is fully working, and will also be merged. PR: https://github.com/AliceO2Group/AliceO2/pull/15318 . For details, see S&C meeting of Monday. Will present in more details at the TPC Calib meeting next week.
Other topics:
- GCC bump - Status?
EPN GPU Topics:
-
10:20
→
10:25
TPC ML Clustering 5mSpeaker: Christian Sonnabend (CERN)
- 10:25 → 10:30
-
10:30
→
10:35
Efficient Data Structures 5mSpeaker: Dr Oliver Gregor Rietmann (CERN)
Next Generation Triggers
- Implementing feedback from CHEP 2026 processdings
- Next week Monday to Wednesday, I am at the NGT Workshop: https://indico.cern.ch/event/1662909/
O2 CI Pipeline
On every commit on the
devbranch:- Build non-deterministic and benchmark
- Plot the benchmarks with the different commits on the x-axis
Possible implementation:
- Do this in the dev branch via a pipeline that dumps the benchmark results
- Use a manually triggered pipeline that fetches the results from Step 2 and plots them.
O2 TPC Track Reconstruction
- Working on this fork of the AliceO2 repo, with a CI pipeline:
- Running on NGT hardware with 4 different GPUs (Nvidia and AMD)
- Extended CI-pipline to fail if GPU.out changes
- Implemented SoA in SectorTracker: GPUTPCBaseTrackParam, GPUTPCTrackParam, GPUTPCTracklet, GPUTPCTrack
- Adressing feedback of CHEP 2026 review
- Merged newest
devbranch into my branches - Using newest version of dependencies from CVMFS
- Make better use of SoA to improve performance
- TrackletConstructor
- TrackletSelector
- ExtrapolationTracking
- Select extrapolation candidates during TrackletSelector and process only those in ExtraploationTracking
- Storing left and right extrapolation candidates separately
- On MI300x we see the best improvement (6% Wall time), the only one with wavefront size 64.
- Old: middle sector waits for left and right. New: only wait for right neighbor.
- Drawback: I had to make the data of the other sectors readable through a sectors processor.




- Next Steps
- Generate
GPUParameters.csvfile for the newest changes, maybe using a CI pipeline? - Schedule the two TPC rings to disjoint streams and/or use 9 streams.
- Generate
-
10:40
→
10:45
TPC Clusterization / OpenCL / Highly Ionizing Particles 5mSpeaker: Felix Weiglhofer (CERN)
OpenCL
No news.
GPU Servers
- CI Server: Test for deterministic mode done (https://github.com/alisw/alidist/pull/6423)
- Added
--PROCdebugDumpChecksumto compute checksum of dumped data on the fly - So we need to compare only hashes of dumps
- Testdata stored on alien:
/alice/cern.ch/user/a/alienci/gpuci/events - Also allows to reproduce test locally
- Test works on CI machine
- TODOs:
- Try David's comment to also recreate remaining dump files
- Need a way to create and pass alien tokens to the CI jobs
- Added
Highly Ionizing Particles
- Adjusted fanout behavior:
- Old, more aggressive fanout: lost 0.8% tracks
- +-2 timebins and pads: +0.1% tracks
- no fanout: +0.3% tracks
Tried plotting dE/dx (qTot) and rejected tracks (no filter, old filter, +- 2 fanout, no fanout):

- CI Server: Test for deterministic mode done (https://github.com/alisw/alidist/pull/6423)
-
10:45
→
10:50
ITS Tracking 5mSpeakers: Felix Schlepper (CERN, Heidelberg University (DE)), Gabriele Cimador (CERN, Università and INFN Torino), Matteo Concas (CERN)
Reprocessed run 572622 (LHC26ak, Pb–Pb at 23 kHz) with new GPU vertexer




Next steps
- Saw -1.3% yield for D0s and -0.46% yield for K0s
-
One flag removes the events: kNoSameBunchPileup
-
It rejects a collision if another reconstructed collision is assigned to the same bunch crossing
-
The new vertexer finds more low-multiplicity vertices (5–49 contributors). Some get assigned to the bunch crossing of a nearby central collision, so the central collision looks like same-bunch pile-up and is thrown away
- Investigate this issue
- Tune params better for UPC
- Use GPU Tuner without RTC
-
10:50
→
10:55
System Run Coordination Topics 5mSpeaker: Ernst Hellbar (CERN)
-
10:00
→
10:20