Alice Weekly Meeting: Software for Hardware Accelerators

→ Europe/Zurich
Zoom Meeting ID
61230224927
Host
David Rohr
Useful links
Join via phone
Zoom URL
    • 15:30 → 15:50
      Discussion 20m
      Speaker: David Rohr (CERN)

      Color code: (critical, news from this week: blue, news from last week: purple, no news: black)

      Sync reconstruction

       

      Async reconstruction

      • Need to investigate short GPU stall problem.
      • Limiting factor for pp workflow is now the TPC time series, which is to slow and creates backpressure (costs ~20% performance on EPNs). Enabled multi-threading as recommended by Matthias - need to check if it works.
      • Will tune existing 16-core settings, add a SITEARCH for 16core CPU, and 16coreCPU + generic NVIDIA / AMD GPU, like for 8 core.
      • Will retune EPN async workflow for TPC + ITS on GPU on 2025 data.

       

      GPU ROCm / compiler topics:

      • Problem with building ONNXRuntime with MigraphX support.
      • Need to find a way to build ONNXRuntime with support for CUDA and for ROCm.
      • Try to find a better solution for the problem with __device__ inline functions leaking symbols in the host code.
      • As discussed with Giulio, want to attempt to move CUDA / ROCm from system installation to alidist.

       

      TPC / GPU Processing 

      • WIP: Use alignas() or find a better solution to fix alignment of monte carlo labels: https://its.cern.ch/jira/browse/O2-5314
      • Need to check the problem with ONNX external memory allocator.
      • Next high priority topic: Improvements for cluster sharing and cluster attachment at lower TPC pad rows. PR: https://github.com/AliceO2Group/AliceO2/pull/14542
      • Check for unnecessary f64 instructions in GPU code.
      • Buffer overrun is real, not due to wrong estimation. Bug somewhere in TPC digit processing or clusterization. Still need to check.

       

      Other topics:

      • Need to bump ONNXRuntime to 1.24, Giulio is checking, needed for ROCm 7.2 - Status?
      • GCC bump - Status?

       

      EPN GPU Topics:

      • Still need to do long-term stress test with ROCm 7.14.
    • 15:50 → 15:55
      TPC ML Clustering 5m
      Speaker: Christian Sonnabend (CERN, Heidelberg University (DE))
    • 15:55 → 16:00
      GPU Parameter Optimizations 5m
      Speaker: Gabriele Cimador (CERN, Università and INFN Torino)
    • 16:00 → 16:05
      Efficient Data Structures 5m
      Speaker: Dr Oliver Gregor Rietmann (CERN)
       

      NextGenTrigger Task 1.7

      • Supervising a student to work an a CMS clustering algorithm
        • Integrated our idea in the CLUEstering code
          • Correctness: We get the same results as with the old algorithm in 2D
        • Next steps:
          • Generalize to nD
          • Improve performance using Monte-Carlo quadrature

      Implement NGT SoA Code in O2 standalone benchmark

      • Working on this fork of the AliceO2 repo, with a CI pipeline:
        • Running on NGT hardware with 4 different GPUs (Nvidia and AMD)
        • Extended CI-pipline to fail if GPU.out changes
      • Implemented SoA in:
        • SectorTracker
          • GPUTPCBaseTrackParam
          • GPUTPCTrackParam
          • GPUTPCTracklet
          • GPUTPCTrack
      • CHEP2026 proceedings
      • Make better use of SoA to improve performance
        • Sorted Tracklets by lastRow and (lastRow - firstRow) to counter warp divergence.
        • For TrackletSelector on (partitionned) Nvidia H100:
        • metric old new
          Avg. Active Threads Per Warp  7.27 / 32 (23%)  13.43 / 32 (42%)
          Non-fused FP32 instructions executed 1.05M 0.59M
          Bytes utilized per 32-byte sector 5.1 4.1
           Uncoalesced global access "Est. Speedup" 66.5% 77.3%
      • Next Steps:
        • Fix worsening of the coalescing (due to TrackletRowHits).
        • Try different way of allocating tracklets.
        • Generate GPU parameters .csv file for SoA and to see if the optimal parameters are different
    • 16:10 → 16:15
      TPC Clusterization / OpenCL / Highly Ionizing Particles 5m
      Speaker: Felix Weiglhofer (CERN)

      OpenCL

      No news.

      GPU Servers

      • CI Server:
        • Deployed tests successfully locally via Nomad
        • Test recipes merged in alidist
        • Giulio setup O2 integration, GPU tests should start to show up on PRs

       

      Highly Ionizing Particles

      Started to implement changes from TPC discussion:

      • Working on fixing the time position

       

       Focusing on ML4Jets talk currently.

    • 16:15 → 16:20
      ITS Tracking 5m
      Speakers: Felix Schlepper (CERN, Heidelberg University (DE)), Gabriele Cimador (CERN, Università and INFN Torino), Matteo Concas (CERN)

      Felix

      Worked on speeding up CPU/GPU ITS tracking (same idea as Gabriele presented last week: buffer size estimation).

      CPU:

      48 kHz Pb-Pb 23% faster (here the size estimates are spilling more often due to charged-particle multiplicity fluctuations)
      750 kHz pp 30% faster

      GPU:

    • 16:20 → 16:25
      System Run Coordination Topics 5m
      Speaker: Ernst Hellbar