Focus on:  Hide Contributions
Login

BiLD-Dev

→ Europe/Zurich
2/R-014 (CERN)

2/R-014

CERN

10
Show room on map
Federico Stagni (CERN)
Description

Bi-Weekly "Loyal" DIRAC developers meeting. And, following, the LHCbDIRAC developers meeting.

Zoom: BiLD
https://cern.zoom.us/j/62504856418?pwd=TU1kb01SOFFpSDBJeWVBdU9qemVXQT09

Meeting ID: 62504856418
Passcode: 12345678
 

Live notes
BiLD
Zoom Meeting ID
62504856418
Host
Federico Stagni
Useful links
Join via phone
Zoom URL
Hide
 
 

BiLD – 13/11/2025

At CERN: Federico, Christophe, Christopher, Ryun, Alexandre
On Zoom: Ueda, Daniela, Hideki, Alexei, Natthan, Andrei, Mazen, Jorge, Xiaomei, Dhiraj
Apologies:

Previous meetings

  • Last BiLD was 3 weeks ago

DIRAC communities roundtable

LHCb:

Federico+Christopher+Christophe+Alexandre+Ryun

  • Updated to proper (LHCb)DIRAC and (lhcb)DiracX releases :tada:
  • We finally solved issues with CS becoming unavailable
    • “solved” by having a load balancer (external to DIRAC) in front of the CS slaves. Just a machine installing HAProxy. DIRAC’s balancer exposes issues when there’s a high load
      • Andrei+Xiaomei saw in Juno similar issue, maybe this could be applied there too

Belle2

Hideki, Ueda, Dhiraj

  • Scalability issues: MySQL. Probably caused by many short jobs
    • Andrei there are plots for jobs/second executed that could be useful for you
    • there’s an overhead for starting jobs so should avoid running 1-minute jobs
  • Switching VOMS, also KEK CAs.
  • from previous meeting Found some stomp related issues, will check if already signaled, if not will create a new one

Juno+BES3:

Xiaomei

  • Some bottleneck issues lately, many small jobs hammering the CS and MySQL, and the disk where Sandboxes are located
    • Andrei added few CS slaves
    • For Sandboxes --> the main way forward is use DiracX and S3 for sandbox store
  • Forced to use Multi-hop. Mostly successfull, but issues reported
    • Christophe saw the issues, will look at it soon

EGI

Andrei

  • Smooth running. Chasing sites to update the configuration (VOMS, tokens, etc)
  • Created GreenDIRAC extension for keeping related developments: GreenReportingAgent, energy consumption by jobs tool (Mazen)
    • Submitting energy metrics for each job into the DB
  • Started a GreenSiteDirector
    • work in progress, for submitting to sites that have a lower energy footprint
  • Got functional D9/Dx0.0.1 certification installation (Bertrand):
    • DIRAC9 host + 2 hosts k3s cluster for DiracX + S3/minIO
      • Chris+Federico should not run on 2. Better 1 or 3
    • Opensearch with certificate authentication
      • will contribute to DiracX for this
    • IAM dev IdP, failed with dteam IAM, Check-In in progress
    • wants to add also a MQ

GridPP:

Daniela, Simon

  • No change in production.
  • DIRACx investigation slow, mainly due to Daniela being pre-occupied with other things.
  • Could we at some point in the meeting have a quick look at: https://github.com/DIRACGrid/diracx/discussions/700 (“CS repo directory problems in container deployment”)

Topics from GitHub discussions and bots

  • only un-answered DIRAC and DiracX topics with discussion updates:
    • NTR

Releases

DIRAC

  • v8.0.76
    • no new releases since July
  • v9.0.6
    • Configuration
      • CHANGE: (#8353) add PreferredURLPatterns to pilot.json
    • FrameworkSystem
      • NEW: (#8328) allow for multiple client per provider
    • DataManagementSystem
      • NEW: (#8328) FTS3 gets tokens directly

Issues (still there?):

DiracX

Release planning, tests and certification

  • Shall we use immutable releases?

    • done for DIRAC and DiracX

  • Upgrade to v9+0.0.1:

    • Federico I have not touched the guide in the last 2 or 3 weeks, any more advise? Anyone else tried it out?

  • Certification machines

    • Federico I have been asked to add CMS VO there, with just 1 user.
      * dteam will be enough
      • Andrei CMS could go in EGI, even

  • Next hackathon(s)

    • Next week, for real this time

DIRAC projects

DIRAC:

Issues by milestone:

Other issues:

PRs discussed:

WebApp:

  • Created v6.0.0 version

Pilot:

  • PR for removing Python2 support re-created (previous one introduced a bug)
    • will be merged into master after a DIRAC(X) hackathon
    • from previous meeting if you really want to keep py2 support, there’a tag
  • PR for using the PreferredURL pattern
    • ready (finally also tests are all passing)

DIRACOS:

  • NTR

Documentation:

management

  • Having again issues for /cvmfs/dirac.egi.eu (the syncing was not happening, ticketed, etc.). No monitoring on their side, urgent need to move out.
  • from previous meeting new /cvmfs/dirac.cern.ch repository created – CERN ticket
    • action on cburr to populate it (using LHCb “machinery”)

DB12

  • NTR
  • from previous meeting Federico “I asked Igor if he wanted to become one if not the main maintainer”
  • from previous meeting We should agree on a strategy on how to do things here, as PRs can’t just be merged:
    • there are reports created that depend from it, merging/releasing on a random Tuesday is not the way to go
    • can we have parallel benchmarks? not clear actually if that’s doable
    • Federico maybe create a “2026” release?
      • we should see if there would be time for that

Rucio

Tests

  • NTR

DiracX:

Issues and Discussions

PRs discussed:

CWL prototype

DiracX-charts:

DiracX-web:

  • NTR

Next appointments

AOB

  • It’s again CHEP abstract submission time, deadline 19th December

    • Dirac(X) “state of affairs” abstract? Other suggestions?
    • Federico will write down an abstract but will not go.

  • ISIMA forum 20th-21st November (students will be LHCb-hosted). Time to write down project proposals – ideas?

    • Federico+ChrisB+Alexandre writing down a proposal for a new Jobs Matching mechanism for DiracX: https://codimd.web.cern.ch/eOU4kV3jRTSSJq43FXvhwg#
      • idea would be that student prepares a prototype for stress tests (e.g. 10M jobs in queue, how fast can we match?)
      • (new) description of job requirements to be used will need your suggestion
    • Any non-LHCb have a proposal and want to be a non-official supervisor?

  • from previous meeting DIRAC as an “HSF affiliated project” : https://hepsoftwarefoundation.org/projects/affiliated.html

    • 13th Nov 2025 no news, but in the meantime this was presented at few forums (…). In the meantime Ruslan restarted the evaluation.

LHCbDIRAC

  • from previous meeting The CS in lhcbdiracx has not been updated lately because of new fields added which do not pass the validation
    • so no new users in the last 10 days
    • protection from that is in the works (issue in DIRAC attached to Natthan)
  • from previous meeting The CS overloading is still on-the-edge, needs Pilots to respect the PreferredURL regex
    *
  • Several MRs in the system, reminders to update the existing ones and check what’s actually ready for merge
  • Bookkeeping
  • LHCb abstracts for CHEP go in https://codimd.web.cern.ch/4mOg3RgsT1eVTnb4KT3Tuw?edit
 
There are minutes attached to this event. Show them.
    • 10:00 → 10:10
      Items from Previous BiLD-Dev 10m
    • 10:10 → 10:20
      DIRAC Communities roundtable 10m
    • 10:20 → 10:30
      DIRAC releases 10m
    • 10:30 → 10:55
      DIRAC projects 25m
      • DIRAC
      • WebApp
      • Pilot
      • DIRACOS2
      • VMDIRAC
      • Documentation
      • OAuth2
      • DiracX
      • other externals (include Rucio)
    • 10:55 → 11:00
      Release planning, tests and certification 5m
    • 11:00 → 11:15
      Weekly development(s) focus 15m
    • 11:15 → 11:25
      AOB
      Convener: Federico Stagni (CERN)
    • 11:25 → 11:40
      LHCbDIRAC 15m