BiLD – 23/10/2025

At CERN: Federico, Christopher, Ryun, Alexandre
On Zoom: Ueda, Igor, Daniela, Janusz, Hideki, Alexei, Natthan, Andrei, Mazen, Alexei, Jorge
Apologies: Andrè

Previous meetings

Last BiLD was 2 weeks ago

DIRAC communities roundtable

LHCb:

Federico+Christopher+Alexandre+Ryun

Issues with CS becoming unavailable have been solved (for the moment).
Will update to last versions next week (preparing the updates now)

Belle2

Hideki, Ueda

Found some stomp related issues, will check if already signaled, if not will create a new one
Some old sites still using python 2 (still using EL7)
- PilotWrapper should first look for py3

CTA:

Natthan

NTR
Working on CWL

GridPP:

Daniela, Simon, Janusz

No news in production – still on v8.0.74
v9.0.1 passed the first very basic test: no DiracX, no S3 and one sucessful job at Imperial
Fixes went into v9.0.2 and Daniela can confirm this still works.

EGI

Andrei

Smooth in production, 8.0.75
Created GreenDIRAC extension for keeping related developments: GreenReportingAgent, energy consumption by jobs tool (Mazen)
Got functional D9/Dx0.0.1 certification installation (Bertrand):
- DIRAC9 host + 2 hosts k3s cluster for DiracX + S3/minIO
  - Chris+Federico should not run on 2. Better 1 or 3
- Opensearch with certificate authentication
  - will contribute to DiracX for this
- IAM dev IdP, failed with dteam IAM, Check-In in progress

JINR:

Igor

NTR
Working on consistency check between DIRAC FC and EOS storage. First result for BM@N:

EOS records (files+dirs):  1340323
DIRAC records (files+dirs):  1340280

Files: DIRAC + / EOS - :  50
Files: DIRAC - / EOS + :  64

Dirs: DIRAC + / EOS - :  0
Dirs: DIRAC - / EOS + :  29

Topics from GitHub discussions and bots

only un-answered DIRAC and DiracX topics with discussion updates:
- NTR

Releases

DIRAC

v8.0.76
- Nothing new since last meeting
v9.0.2
- “just fixes” #worksforgridpp
v9.0.3
- DiracX
  - NEW: (#8347) Depend on released versions of DiracX
  - NEW: (#8347) Depend on diracx-cli to ease management of DiracX CS conversion
- FrameworkSystem
  - CHANGE: (#8324) Remove Registry/group option VOMSVO (simply use “VO”)
v9.0.4
- because of network glitch

Issues (still there?):

https://github.com/conda-forge/dirac-grid-feedstock/issues/170 https://github.com/conda-forge/dirac-grid-feedstock/issues/169
- cburr will take care of these ASAP

diracx

actually, v0.0.2 also created
also diracx-web tagged

Release planning, tests and certification

reminders:
- for DIRAC there’s still only integration branch (~same for WebApp)
  - we’ll keep using the integration branch for now
Upgrade to v9+0.0.1
- the wiki has been again updated several times in the last 2 weeks, after remarks from GridPP.
Certification machines
- Federico not updated (no time). I will try in the next days so that we have a hackathon next week
Next hackathon(s)
- Federico will call one once the cert machines are updated

DIRAC projects

DIRAC:

Issues by milestone:

v8.0:
- Add config option to prevent some groups from cleaning transformations
  - Federico assigned to me, can I assign to someone from Juno?
- CS Slave refreshing from itselfs
  - Questions for Juno to be answered
- Don’t block the transformation system if owner is removed
  - Xiaomei to give more details
- RucioFileCatalogClient.setMetadata returns empty result on success
v9.0:
- Job Pre/Post processing flexibility
  - actually working for this in the CWL prototype
  - Andrei we would actually need to add plugins
v9.1:
- NTR

Other issues:

Get infos from CRIC
- any volunteer?
Make the PushJobAgent compatible with more CEs (HTCondor? SSH?)
*
from previous meeting Proposal for input data resolution format
- also cited in https://github.com/aldbr/dirac-cwl-proto/issues/25
from previous meeting Replacement for BDII2CSAgent #8194
- “nice” discussion
- 17th July
  - recent answers in the GGUS ticket above from AP seems to point in the right direction
  - nevertheless, Federico is trying to grab info on if we can do a “CEs” crawler ourselves. ARC CEs seem to provide the necessary information, issues are from HTCondorCEs
- This has now been discussed in the GridPP project management board. Conclusion was unfortunately “input from GOCDB team needed”
- 09 Oct 2025 GridPP ops team asked UK sites to implement Alessandro’s suggestion. The following issues were observed:
  - It’s not documented (beyond a soon-to-be-archived broadcast)
  - Free form encourages typos and misremembered VO names (vo.na62.gridpp.ac.uk vs na62.vo.gridpp.ac.uk) – this would not happen with predefined scopes
  - RAL as the Tier 1 hit the length limit for extension as it supports too many VOs
  - Sigh.
    - Last message from Federico:
    For those of you who are running a HTCondor CE, would you mind investigating if the content of auth-map + accounting-map files could be made public? Maybe we are lucky and it’s a trivial thing to be done.
    - Daniela This has gone to the HTCondor developers via RAL, but so far no answer.

PRs discussed:

PoolCE and RAM (issue raised in https://github.com/DIRACGrid/DIRAC/issues/7853#issuecomment-2948565279): https://github.com/DIRACGrid/DIRAC/pull/8232
- Federico made progresses, ready to be reviewed since last week. The logic in there shoud be checked
- Andrei matching done with tags, which is not ideal – there is also a limit on the number of tags that we can use.
- cburr should really use MB instead of GB
V9.0 feat fts direct tokens
- requires some coordination – Christophe to sort these ones out
from previous meeting add scitag support
- “re-written” and ready to merge

WebApp:

not Created v6.0.0 version – issue with pipeline, will check later on

Pilot:

PR for removing Python2 support merged in devel branch
- will be merged into master after a DIRAC(X) hackathon
- if you really want to keep py2 support, there’a tag
- At some point can also merge the equivalent DIRAC PR which will effectively also render the p2_eol tag useless
from previous meeting Pilot migration
- diracx pilot route
from previous meeting PR feat: Adding JWT support alongside X509 auth
- the new Pilot command can call directly the route, no need to use the CLI
- the integration tests for this will be set up once diracx is updated with the connected diracx PR

DIRACOS:

Documentation:

DiracX doc PRs connected in different repos. Federico TBH I am not sure splitting the doc on separate repos was the best idea (also: see Trigger a readthedocs build after merging PRs), but now it’s done, let’s keep it
from previous meeting tasks: https://github.com/search?q=org%3ADIRACGrid is%3Aissue state%3Aopen label%3Adocumentation &type=issues
- including some for diracx-charts and diracx-web

management

from previous meeting new /cvmfs/dirac.cern.ch repository created – CERN ticket
- action on cburr to populate it (using LHCb “machinery”)

DB12

Federico “I asked Igor if he wanted to become one if not the main maintainer”
- revived https://github.com/DIRACGrid/DB12/issues/9
- create https://github.com/DIRACGrid/DB12/issues/15 and made a suggestion that can be accepted
- PR https://github.com/DIRACGrid/DB12/pull/16 created – will wait for Alexandre for review
We should agree on a strategy on how to do things here, as PRs can’t just be merged:
- there are reports created that depend from it, merging/releasing on a random Tuesday is not the way to go
- can we have parallel benchmarks? not clear actually if that’s doable
- Federico maybe create a “2026” release?
  - we should see if there would be time for that

Rucio

from previous meeting CTAO submitted draft PR https://github.com/DIRACGrid/DIRAC/pull/8327 and issue https://github.com/DIRACGrid/dirac/issues/8331
- 23 Oct 25 Federico I asked Cedric to review these and other Rucio issues/PRs, but “no time”.

Tests

DiracX:

dependabot alert for web
- N interesting TR
Releases are created thanks to release-please tool
- diracx-web was using this tool already
- cburr maybe what will NOT be used at the end, so do not look into it yet

Issues

There are several “Good first issues” looking for volunteers

PRs discussed:

All existing PRs are either Draft or have been there for some time
from previous meeting Alembic and triggers
- asked Cedric to review it
from previous meeting Add pilot management: create/delete/patch and query #570
- PR looks OK-ish to merge (should not affect existing running code)
from previous meeting feat: deploy gubbins images #527
- Federico I rebased it but still failing

CWL prototype

“moving” with full plan in https://codimd.web.cern.ch/tUL0IUMKT56tsNv3dAf65w?view.
- Alexandre “Too early to report for today”

DiracX-charts:

1.0.0 released
Moved to fixed versions in diracx/values.yaml and added automatic chart version bumping on DiracX releases
- Similarly should be done for diracx-web

DiracX-web:

PRs needing merging?

Next appointments

Meetings:
- Maybe DIRAC(X) cert hackathon in 1 week
- BiLD: in 2 weeks
WS/hackathons/conferences:
- DiracX hackathon 21st-22nd January 2026: https://indico.cern.ch/event/1582395/

AOB

It’s again CHEP abstract submission time, deadline 19th December
- Dirac(X) “state of affairs” abstract? Other suggestions?
- who plans to go?
ISIMA forum 20th-21st November (students will be LHCb-hosted). Time to write down project proposals – ideas?
- do you want to be a non-official supervisor?
from previous meeting DIRAC as an “HSF affiliated project” : https://hepsoftwarefoundation.org/projects/affiliated.html
- 23 Oct 2025 No further updates

LHCbDIRAC

Some MRs created/merged/hotfixed by Ryun
The CS in lhcbdiracx has not been updated lately because of new fields added which do not pass the validation
- so no new users in the last 10 days
- protection from that is in the works (issue in DIRAC attached to Natthan)
The CS overloading is still on-the-edge, needs Pilots to respect the PreferredURL regex
- propagate to pilot.json and then pilot.cfg and then OK (?)
Several MRs in the system, reminders to update the existing ones and check what’s actually ready for merge
Bookkeeping
- We should change the passwords of our dirac users - new should be at least 15 characters long
- from previous meeting
  - First priority is the repartition, that will happen either the last week of October or the first of November. After that, the existing PRs will be evaluated
- add materialized view for processing paths optimization
  - this should be merged first and the next ones use it
  - Alexey it does not work for me, see comment in MR
- reworked addProcessing
- reworked getproductionporcpassname
LHCb abstracts for CHEP go in https://codimd.web.cern.ch/4mOg3RgsT1eVTnb4KT3Tuw?edit