BiLD – 17/07/2025
At CERN: Federico, Theau, Robin, Cedric, Alexandre, Andrè, Christopher, Ryunosuke
On Zoom: Andrei, Daniela, Simon, Janusz, Jorge, Xiaomei, Hideki, Ueda, Natthan, Alexei, Dhiraj
Apologies:
Previous meetings
- Last BiLD was 3 weeks ago
DIRAC communities roundtable
LHCb:
Federico+Christophe+Alexandre+Vladimir+Robin+Theau
- We found several millions of jobs in JobDB that were part of Transformations in status “Archived” or “Cleaned”. The source is probably the TransformationCleaningAgent [TBC] – PR with fixes: https://github.com/DIRACGrid/DIRAC/pull/8244
- scripts to
- (run them one after the other)
Belle2
Cedric, Hideki, Ueda, Dhiraj
- WebApp was down due too much memory consumption (maybe due to extension, TBC)
- BelleRAWDIRAC migrating to v9
CLIC
Andrè
IHEP
Xiaomei
- from previous meeting Started using chatbot for DIRAC – using llamaindex
- setting up a propotype, for the moment just answering questions about DIRAC (documentation)
- with a DB.
- One student working on it
- Alexandre independently, started building a small MCP server for diracx
- 26th June Junau showed 2 slides with the prototype they have done. The work is mostly about documentation.
- Federico the 2 shuld be integrated one to the other
- Alexandre let’s discuss on mattermost, I have also few ideas
- 17th July ready to combine the work, student will try to push to Alexandre’s repository (not yet done)
EGI:
Andrei
- Started work on the agent to send “energy/CO2” records (from DIRAC, but not only) to the GreenDIGIT MetricsDB service. Working together with Henryk.
- From ops pov, usual struggle for changing the certificates
CTA:
Natthan
GridPP:
Daniela, Simon, Janusz
- Nothing to report in production. Submitted some minor patches for “GridPP” usage cases, should not affect anyone else.
Topics from GitHub discussions and bots
- only un-answered DIRAC and DiracX topics with discussion updates:
- Filename for DFC has limit of 128 characters
- “extend freely” from the DIRAC pov, but it’s a slippery slope, and eventually you will find issues with the storage (e.g. EOS – where there’s a 1024 characters limit).
- putting metadata in the filename is in general wrong
- FTS also puts a limitation (at least in DIRAC)
- Andrè linux has 256 limit
Releases
DIRAC
- v8.0.76
- ConfigurationSystem
- FIX: (#8255) Stops VOMS2CSSynchronizer crashing on robot DNs that don’t follow the CERN pattern.
- Resources
- CHANGE: (#8214) SLURM plugin now supports the WholeNode options
- ResourceStatusSystem
- CHANGE: (#8213) Added a token expiry option to dirac-rss-set-status and dirac-admin-allow/ban-site commands.
- v9
- Last pre-release contains several changes to the RPCs. Few bugs too. Now needs:
diracx
- v0.0.1a46 created 2 days ago
- We should remember to generate a new release every time there’s a change in the (generated) client
Release planning, tests and certification
The flowcharts below details the interactions and constrains between some of the existing PRs (in DIRAC, diracx, and Pilot) for v9+0.1 and v9.1+0.2 versions.
9.1+0.2
DIRAC_8196 == Add DiracX support into the database.
9.1
pilot_248 == feat: Adding JWT support alongside X509 auth
diracx_421 == Adding pilot registrations and authentification (Router)
the new DB table for pilot auth
0.2
DIRAC_8097 == feat: added foreign keys to PilotAgentsDB
9.1
diracx_550 == Robin pilot logging
0.2
pilot_260 == Robin add pilot logging
diracx_468 == Splitted the job endpoints to separate Pilots and Users
0.2
9.0+0.1
DIRAC_8233 == feat: Migrating pilot client toward DiracX
9.0
DIRAC_8248 == Improve CI to include legacy adaptor
9.0
diracx_570 == Add pilot management: create/delete/patch and query
0.1
diracx_469 == fix: job admins cannot see jobs from other users
0.1
diracx_593 == Add job monitoring utilities to the routers
0.1
diracx_454 == extra checking for naive datetimes
0.1
DIRAC_8260 == Update job monitoring to use DiracX
9.0
We only looked at the v9+0.1 chart. Most of the PRs involved are from Robin.
-
Certification machines
-
Next hackathon(s)
- As soon as we’ll get “proper” releases
DIRAC projects
DIRAC:
Issues by milestone:
Other issues:
- from previous meeting Replacement for BDII2CSAgent #8194
- “nice” discussion

- Daniela is going to check if (as she suspects) GOCDB is actually underwritten by GridPP. Maybe (can’t promise) “finances” might be an angle.
- 26th June no updates in ticket in the last 2 or 3 weeks
- 17th July
- recent answers in the GGUS ticket above from AP seems to point in the right direction
- nevertheless, Federico is trying to grab info on if we can do a “CEs” crawler ourselves. ARC CEs seem to provide the necessary information, issues are from HTCondorCEs
- Last message from Federico:
For those of you who are running a HTCondor CE, would you mind investigating if the content of auth-map + accounting-map files could be made public? Maybe we are lucky and it’s a trivial thing to be done.
- Daniela will check the above
PRs discussed:
WebApp:
- New alhpa version created (nothing important)
Pilot:
DIRACOS:
Documentation:
management
- from previous meeting new
/cvmfs/dirac.cern.ch repository created – CERN ticket
- question on if it could be
/cvmfs/diracgrid.org – potentially, but that would not be automounted
- action on @cburr to populate it (using LHCb “machinery”)
- Christophe I will propose some ideas on the structure in one or 2 BiLD meetings
- Federico You do not have full flexibility, as the LHCb structure has been copied to DIRAC and IIRC Belle2 too
DB12
Rucio
Tests
- Robin improved
integration_tests.py to add DiracX service(s) – basically for testing the legacy adaptors – almost done
- from previous meeting Federico Started adding Rucio to Dirac integration tests
DiracX:
Issues
- Architecture design principles:
- from previous meeting Federico wrote down an “epic” with 5 sub-tasks: https://github.com/DIRACGrid/DIRACx/issues/562 for the DiracX accounting/monitoring after consulting with CERN experts
- 17th July We gave Ewoud (CERN IT) a task to try this one out on a 6-months popularity data (from LHCb). Current stoppers:
- Grafana does not yet support plotting of OpenSearch rolled-up data (seems to be OK with data from ElasticSearch though, so probably something that will eventually be sorted out)
- More worringly, rollup jobs do not seem to work on DataStreams (work only on standard indices)
- Comments to https://github.com/DIRACGrid/diracx/issues/585#issuecomment-3051642916
- Federico DELETE Jobs should not be there – mark as
JobStatus.DELETED instead
- Andrei is the fact that we do not use RESTful going to have repercussion on the tools?
PRs discussed:
DiracX-charts:
DiracX-web:
Next appointments
AOB
LHCbDIRAC
- Release:
- Bookkeeping MRs:
- from previous meeting StorageReport app available in https://lhcbdiracx-cert.app.cern.ch/
- few minor issues still to be sorted out before accepting the PR
- 26th June pending some code re-organization