BiLD – 05/06/2025

At CERN: Federico, Christophe, Alexandre, Theau
On Zoom: Daniela, Simon, Janusz, Robin, Jorge, Ryunosuke, Dhiraj, Xiaomei, Cedric (briefly)
Apologies:

Previous meetings

Last BiLD was 3 weeks ago

DIRAC communities roundtable

LHCb:

Federico+Christophe+Christopher+Alexandre+Ryun+Vladimir+Alexey+Robin+Theau

Belle2

Hideki, Ueda, Dhiraj

Planning to migrate to OpenSearch for JobParameters.
Planning to deploy v9 for certification, issues with extension
Dhiraj good progress for RAWBelleDIRAC, which migrated to v9 and using diracx web from k3s. All seems OK
- performing tests

IHEP

Xiaomei

Started using chatbot for DIRAC – using llamaindex
- setting up a propotype, for the moment just answering questions about DIRAC (documentation)
  - with a DB.
  - One student working on it
- Alexendre independently, started building a small MCP server for diracx
  - https://github.com/aldbr/dirac-agentic
- Federico I will bring this one up again at the next meeting, but these initiatives should not diverge

GridPP:

Daniela, Simon, Janusz

No change in prod, running v8.0.74.
Daniela finally managed to convey her insights (?) from the January hackathon to Simon, so we currently have both of our developement servers using one opensearch instance for testing and having at least nominally RSS enabled.
These test also triggered this pull request, which from the GridPP point of view just re-instates previous behaviour:
- https://github.com/DIRACGrid/DIRAC/pull/8213
- I know this is not how LHCb does it, but I don’t think 20+ lines of config for a simple on/off switch is appropriate either. #worksforgridpp
  - Federico add the --days switch to the dirac-admin-ban-* scripts and I will accept this
We haven’t written it up yet, but we are still hoping to use one IAM server in multi-VO mode, using groups as a stand in for VOs. There were fairly positive noises from Chris^2 that the VOMS2CSSynchronizer could be adapted (can be an option, clearly) to read out the group membership from IAM as well and then match e.g. GridPP IAM + gridpp -> gridpp VO users, GridPP IAM + lz -> lz VO users when generating the user list per VO for DIRACx.
from previous meeting When logging into the cvmfs uploader I noticed that we use a lot of space for releases that presumably will never be used: Can we e.g. delete v9.0.0a* up to the release LHCb is using ?
- ChrisB CVMFS should be de-duplicating things
  - Simon this does not seem the way RAL has set up CVMFS
    Daniela tried to follow up on this last night, but the cvmfs area didn’t seem to be reachable from the uploader. RAL informed.
- from previous meeting On the other hand, can we please add the latest v8.0 release ? At the moment they stop at 71. (There might be some v8 releases that could go as well.)
  - v8.0.74 now there. But could @Andrei please document the upload procedure in a Wiki or similar, if only to increase the bus factor ?
- 5th June Federico would you be against moving things to /cvmfs/dirac.cern.ch?
  - go ahead

Topics from GitHub discussions and bots

only un-answered DIRAC and DiracX topics with discussion updates:
- On Time Series
  - Ewoud (CERN-IT) is investigating using OpenSearch rollup jobs
    - Further info in https://its.cern.ch/jira/browse/UEM-190
  - Federico I think this could be a long term replacement for the DIRAC accounting. Wrote a program of work: https://codimd.web.cern.ch/aMFLvHgfRn-e9lf92J_DyQ#
    - will make epic+tasks out of it if you are OK with it
    - Christophe check with experts of CERN MONIT team (Borja, Ignacio). Opentelemetry data can also be part of it

DIRAC releases

v8.0.7x
*
v9
*

DIRAC projects

DIRAC:

Issues by milestone:

v8.0:
- NTR
v9.0:
- Nothing open
After v9.0:
- Closed 2 today
- The remaining ones imply DB schema changes.

Other issues:

Replacement for BDII2CSAgent #8194
- “nice” discussion
- Daniela is going to check if (as she suspects) GOCDB is actually underwritten by GridPP. Maybe (can’t promise) “finances” might be an angle.
Federico has questions about Using cgroups to limit job resource usage #7853
- Simon missed those, will reply

PRs discussed:

Federico created [9.0] feat: use summary tables and triggers #8199 as replacement for [9.0] JobDB snapshot: use a summary table for jobs in final state #8181
- Christophe will review properly. We need to make sure to not clog the job updates/insertions
feat (WMS): define setJobStatus in the Client to allow keyword arguments #8215
- Federico can we remove the service?
  - Christophe effectively we should wait for everyone to not use it anymore

WebApp:

Created/merged https://github.com/DIRACGrid/WebAppDIRAC/pull/784 which closed Mix of groups for ReportGeneratorHandler #6877 (and draft CVE)
Someone opened issue about path traversal Windows: https://github.com/DIRACGrid/WebAppDIRAC/issues/779
- closed as not relevant

Pilot:

Pilot logging from Robin
- picked up from Janusz
Pilot migration
- diracx pilot route
from previous meeting PR feat: Adding JWT support alongside X509 auth
- the new Pilot command can call directly the route, no need to use the CLI
- the integration tests for this will be set up once diracx is updated with the connected diracx PR

DIRACOS:

Documentation:

diracgrid.org being revamped. It will have:
- info about the consortium
- generic info, independent from DIRAC and DiracX
- links to DIRAC and DiracX documentation
tasks: https://github.com/DIRACGrid/DIRACx/issues?q=is%3Aissue state%3Aopen label%3Adocumentation
Created the DiracX documentation with mkdocs. Hosted on readthedocs.
- ChrisB registered https://diracx.io/ time ago, linking rtd docs
- skeleton quite alright, the content needs large revision
  - Christophe there’s a large-ish PR which would require attention but now taken by the restart of LHCb data taking
on diracx.io: ChrisB registered it under his name more than a year ago. We do anyway have diracgrid.org (that is also “correctly billed”) so it makes sense to use only that domain. Chris is therefore proposing to move to:
- diracx.diracgrid.org (diracx docs, currently diracx.io)
- charts.diracgrid.org (for the helm chart, would have been destined to be charts.diracx.io if we’d leave it as is)
- then we’d need common access for managing the DNS on diracgrid.org to set the CNAME records.
non-technical (for publication, presentations): how to write “dirac” – options (the selected ones are in bold):
- DIRAC or dirac or Dirac
- DIRACx or diracx or DiracX
- LHCbDIRACx or LHCbdiracx or LHCbDiracX
- GridPP: Can’t comment on the last line, but DIRAC and either DIRACx or DiracX. All lowercase always looks like a command not a product.

OAuth2:

management

diraccfg

DB12

from previous meeting https://github.com/DIRACGrid/DIRAC/issues/7760#issuecomment-2482420604
- Federico proposed to create “alternate” benchmark

Rucio

Someone opened a CVE against the plugin: https://github.com/DIRACGrid/DIRAC/security/advisories/GHSA-mm38-q5fj-99j4
- added Cedric to it

Tests

Robin to improve the (DIRAC-hosted, docker-compose based) integration_tests.py to add DiracX service(s) – basically for testing the legacy adaptors
from previous meeting Federico Started adding Rucio to Dirac integration tests
- –> to Janusz

DiracX:

Road Map : https://github.com/DIRACGrid/diracx/blob/caf66076bc8b623b0282a2f3d7d723b7c45be1a6/docs/roadmap.md?plain=1
- the only thing left to complete before v0.1 is documentation
dependabot alert for web
- NTR

Issues

from previous meeting
- Try to setup release-please (or alternatives … semantic-releases?) #264
  - Alexandre studying semantic-releases. Seems alright, when ready it will be used for all diracx projects
- Breaking the API breaks the web interface #350
  - the current decision is “not yet needed. Maybe never”

PRs discussed:

On Pixi : https://pixi.sh/latest/
- What is pixi? blog post
- Pixi PR
- Federico questions:
  - I see a pixi.toml. I thought pyproject.toml was enough
  - should extensions of diracx all use pixi?
  - should I just remove my local diracx and restart?
from previous meeting Is fix: #448 and smarter datetimes #454 ready?
- Jorge looks OK to me but I can’t know all the implications
- ChrisB It is probably OK, but will try to understand it better
- Federico bonus question: if it’s ready and can be merged, does it mean that we can give it another go at postgres support?
  - Jorge will try to revive his previous PR about postgres with this fix in, for testing purposes
  - 5th June
    - open PR in https://github.com/DIRACGrid/diracx/pull/556
    - wrote notes in https://codimd.web.cern.ch/s/LaoLMvUs0#
    - summary after today’s discussion: we give up PostgreSQL for the time being as we would have too many things to do in DIRAC’s MySQL for fixing how the dates are stored.
Table summaries
- maybe use Alembic?
  - it’s a bit weak reason, but why not

DiracX-charts:

documentation needs to be fully reorganized (the open PR on it should be included then)

DiracX-web:

Theau implemented a way for share app states (dump in JSON, reload through web)
- follow-up issue: https://github.com/DIRACGrid/diracx-web/issues/348
- 5th June the new proposal is to have a DB where to stored snippets for the state. Basically, like the UserProfileDB in DIRAC.

Release planning, tests and certification

Certification machines
- from previous meeting Federico will update it with the latest diracx goodies
Next hackathon(s)
- We should go back at not testing in production (e.g. many changes for pilots from Robin)
from previous meeting Few DB changes being prepared (DIRAC):
- remove the TransformationDB.DataFiles table
- added foreign keys to PilotAgentsDB
- and we might be discovering more… 9.1?

Next appointments

Meetings:
- BiLD: in 2 weeks
WS/hackathons/conferences:
- DIRAC Users’ Workshop
  
  📅 17–20 September 2025 | 📍 IHEP, Beijing, China
  🔗 indico.cern.ch/e/duw11
  * Connect with the DIRAC community
  * Meet the developers
  * Discover DiracX
  * Share your insights
  The workshop will feature talks from users, administrators, and developers — plus a hands-on DiracX Hackathon.
  
  Poster in https://cernbox.cern.ch/remote.php/dav/public-files/1egpqkseV0GvqlO/DIRAC_Poster.pdf – make use of it!

AOB

DIRAC as an “HSF affiliated project” : https://hepsoftwarefoundation.org/projects/affiliated.html
- No news

LHCbDIRAC

from previous meeting StorageReport app available in https://lhcbdiracx-cert.app.cern.ch/
- few minor issues still to be sorted out before accepting the PR