BiLD-Dev

Thursday 26 Jun 2025, 10:00 → 12:00 Europe/Zurich

2/R-014 (CERN)

Federico Stagni (CERN)

Bi-Weekly "Loyal" DIRAC developers meeting. And, following, the LHCbDIRAC developers meeting.

Zoom: BiLD
https://cern.zoom.us/j/62504856418?pwd=TU1kb01SOFFpSDBJeWVBdU9qemVXQT09

Meeting ID: 62504856418
Passcode: 12345678
 

live minutes

 

 

BiLD – 26/06/2025

At CERN: Federico, Theau, Robin, Christopher, Andrè
On Zoom: Alexandre, Christophe, Daniela, Simon, Janusz, Jorge, Ryunosuke, Dhiraj, Xiaomei, Xuantong, Cedric, Hideki, Ueda, Yunau, Natthan
Apologies: Andrei

---

Previous meetings

• Last BiLD was 3 weeks ago

---

DIRAC communities roundtable

LHCb:

Federico+Christophe+Alexandre+Vladimir+Robin+Theau

• NTR

Belle2

Hideki, Ueda, Dhiraj

• Questions about Kubernetes deployement: how to expose the pods
  • Christophe load balancers are provided by kubernetes
  • This is a pure kubernetes “issue”: few info: https://github.com/DIRACGrid/diracx-charts/issues/107

CLIC

Andrè

• NTR

IHEP

Xiaomei, Xuantong, Yunau

• from previous meeting Started using chatbot for DIRAC – using llamaindex
  • setting up a propotype, for the moment just answering questions about DIRAC (documentation)
    • with a DB.
    • One student working on it
  • Alexandre independently, started building a small MCP server for diracx
    • https://github.com/aldbr/dirac-agentic
• 26th June Junau showed 2 slides with the prototype they have done. The work is mostly about documentation.
  • Federico the 2 shuld be integrated one to the other
  • Alexandre let’s discuss on mattermost, I have also few ideas

CTA:

Natthan

• NTR

GridPP:

Daniela, Simon, Janusz

• Nothing to report.

---

Topics from GitHub discussions and bots

• only un-answered DIRAC and DiracX topics with discussion updates:
  • NTR

---

DIRAC releases

• v8.0.75
  • NTR
• v9
  *

---

DIRAC projects

DIRAC:

Issues by milestone:

• v8.0:
  • NTR
• v9.0:
  • 1 open recently: Bad refactoring lead to client needing DB access #8218
• After v9.0:
  • Tasks imply DB schema changes – for v9.1

Other issues:

• Deadlock in JobLogging DB #8224
• from previous meeting Replacement for BDII2CSAgent #8194
  • “nice” discussion
  • Daniela is going to check if (as she suspects) GOCDB is actually underwritten by GridPP. Maybe (can’t promise) “finances” might be an angle.
  • 26th June no updates in ticket in the last 2 or 3 weeks

PRs discussed:

• from previous meeting Federico created [9.0] feat: use summary tables and triggers #8199 as replacement for [9.0] JobDB snapshot: use a summary table for jobs in final state #8181
  • Christophe will review properly. We need to make sure to not clog the job updates/insertions
  • 26th June Federico I hotfixed this in DIRAC certification, fixed a few bugs.
• [8.0] fix (resources): condor commandline call compatible with v24 #8225
  • @asailer wants to review?
• PoolCE and RAM (issue raised in https://github.com/DIRACGrid/DIRAC/issues/7853#issuecomment-2948565279): https://github.com/DIRACGrid/DIRAC/pull/8232

WebApp:

• NTR

Pilot:

• Federico I merged a minor PR, only adapting the integration tests
• Pilot logging from Robin
  • picked up from Janusz
• Pilot migration
  • diracx pilot route
• from previous meeting PR feat: Adding JWT support alongside X509 auth
  • the new Pilot command can call directly the route, no need to use the CLI
  • the integration tests for this will be set up once diracx is updated with the connected diracx PR

DIRACOS:

• NTR

Documentation:

• from previous meeting diracgrid.org being revamped. It will have:
  • info about the consortium
  • generic info, independent from DIRAC and DiracX
  • links to DIRAC and DiracX documentation
• from previous meeting tasks: https://github.com/DIRACGrid/DIRACx/issues?q=is%3Aissue state%3Aopen label%3Adocumentation
• from previous meeting on diracx.io: ChrisB registered it under his name more than a year ago. We do anyway have diracgrid.org (that is also “correctly billed”) so it makes sense to use only that domain. Chris is therefore proposing to move to:
  • diracx.diracgrid.org (diracx docs, currently diracx.io)
  • charts.diracgrid.org (for the helm chart, would have been destined to be charts.diracx.io if we’d leave it as is)
  • then we’d need common access for managing the DNS on diracgrid.org to set the CNAME records.

management

• new /cvmfs/dirac.cern.ch repository created – CERN ticket
  • question on if it could be /cvmfs/diracgrid.org – potentially, but that would not be automounted
  • action on @cburr to populate it (using LHCb “machinery”)
    • Christophe I will propose some ideas on the structure in one or 2 BiLD meetings
    • Federico You do not have full flexibility, as the LHCb structure has been copied to DIRAC and IIRC Belle2 too

DB12

• from previous meeting https://github.com/DIRACGrid/DIRAC/issues/7760#issuecomment-2482420604
  • Federico proposed to create “alternate” benchmark

Rucio

• from previous meeting Someone opened a CVE against the plugin: https://github.com/DIRACGrid/DIRAC/security/advisories/GHSA-mm38-q5fj-99j4
  • 26th June https://github.com/DIRACGrid/DIRAC/pull/8220 for solution – comments in it

Tests

• from previous meeting Robin to improve the (DIRAC-hosted, docker-compose based) integration_tests.py to add DiracX service(s) – basically for testing the legacy adaptors
• from previous meeting Federico Started adding Rucio to Dirac integration tests
  • –> to Janusz

---

DiracX:

• Road Map : https://github.com/DIRACGrid/diracx/blob/caf66076bc8b623b0282a2f3d7d723b7c45be1a6/docs/roadmap.md?plain=1
  • the only thing left to complete before v0.1 is documentation
• dependabot alert for web
  • NTR

Issues

• Federico wrote down an “epic” with 5 sub-tasks: https://github.com/DIRACGrid/DIRACx/issues/562 for the DiracX accounting/monitoring after consulting with CERN experts
• Open question from Robin+Federico: are there thougths about a “failover system” for DiracX?
  • Robin was asking because of Pilot logs
• from previous meeting
  • Try to setup release-please (or alternatives … semantic-releases?) #264
    • Alexandre studying semantic-releases. Seems alright, when ready it will be used for all diracx projects

PRs discussed:

• Add pilot management: create/delete/patch and query #570
  • PR looks OK-ish to merge (should not affect existing running code)
• FastAPI does not allow to disable OAuth in sub-routes – how to do the pilots auth route?
  • https://github.com/DIRACGrid/diracx/pull/570#issuecomment-3000715439
• feat: deploy gubbins images #527
  • waiting for https://github.com/DIRACGrid/diracx-charts/pull/158 to be approved and merged
• Is fix: #448 and smarter datetimes #454 ready?
  • Ryun will try to finalize ASAP

DiracX-charts:

• documentation needs to be fully reorganized (the open PR on it should be included then)
• Federico shall we use stargz-snapshotter for faster loading?

DiracX-web:

• Web user-facing documentation: https://github.com/DIRACGrid/diracx-web/pull/367#pullrequestreview-2950696755
  • Conclusion: leave the user documentation in /docs, read it from the code and present it to the users
• The last 2 PRs are failing a test related to gubbins – what’s the source?
• from previous meeting Theau implemented a way for share app states (dump in JSON, reload through web)
  • follow-up issue: https://github.com/DIRACGrid/diracx-web/issues/348
  • 5th June the new proposal is to have a DB where to stored snippets for the state. Basically, like the UserProfileDB in DIRAC.

---

Release planning, tests and certification

• Certification machines

  • from previous meeting Federico will update it with the latest diracx goodies
    • 26th June Done

• Next hackathon(s)

  • probably in 2 weeks

• from previous meeting Few DB changes being prepared (DIRAC):

  • remove the TransformationDB.DataFiles table
  • added foreign keys to PilotAgentsDB
  • and we might be discovering more… 9.1?

Next appointments

• Meetings:

  • Hackathon: in 2 weeks
  • BiLD: in 3 weeks

• WS/hackathons/conferences:

  • DIRAC Users’ Workshop

    📅 17–20 September 2025 | 📍 IHEP, Beijing, China
    🔗 indico.cern.ch/e/duw11
    * Connect with the DIRAC community
    * Meet the developers
    * Discover DiracX
    * Share your insights
    The workshop will feature talks from users, administrators, and developers — plus a hands-on DiracX Hackathon.

    Poster in https://cernbox.cern.ch/remote.php/dav/public-files/1egpqkseV0GvqlO/DIRAC_Poster.pdf – make use of it!

AOB

• DIRAC as an “HSF affiliated project” : https://hepsoftwarefoundation.org/projects/affiliated.html
  • no news

---

LHCbDIRAC

• Apply https://github.com/DIRACGrid/DIRAC/pull/8202 ?
  • Christophe will apply it
• from previous meeting StorageReport app available in https://lhcbdiracx-cert.app.cern.ch/
  • few minor issues still to be sorted out before accepting the PR
  • 26th June pending some code re-organization
• BKK isn’t happy. Suspiciously close to the installation of the release.

 

10:00 → 10:10 Items from Previous BiLD-Dev

10:10 → 10:20 DIRAC Communities roundtable

10:20 → 10:30 DIRAC releases

10:30 → 10:55 DIRAC projects

• DIRAC
• WebApp
• Pilot
• DIRACOS2
• VMDIRAC
• Documentation
• OAuth2
• DiracX
• other externals (include Rucio)

10:55 → 11:00 Release planning, tests and certification

11:00 → 11:15 Weekly development(s) focus

11:15 → 11:25 AOB

Convener: Federico Stagni (CERN)

11:25 → 11:40 LHCbDIRAC