Choose timezone
Your profile timezone:
BiLD-Dev
Description
Bi-Weekly "Loyal" DIRAC developers meeting. And, following, the LHCbDIRAC developers meeting.
Zoom: BiLD
https://cern.zoom.us/j/62504856418?pwd=TU1kb01SOFFpSDBJeWVBdU9qemVXQT09
Meeting ID: 62504856418
Passcode: 12345678
BiLD
BiLD – 15/05/2025
At CERN: Federico, Christopher, Alexandre, Robin, Theau
On Zoom: Daniela, Simon, Janusz, Jorge, Dhiraj, Natthan
Apologies: Andrei, André
Follow-up from previous meetings
- Last BiLD was 6 weeks ago (extreme summary: Federico presented how LHCb moved to v9+X)
- Last week (5-6 May) we hold a DiracX hackathon here at CERN.
- 14 people on site (7 LHCb, 4 Belle2)
- Main goals:
- writing DiracX documentation, so that we can think of releasing DIRAC v9 and DiracX v0.1
- advance on DiracX migration for Belle2 and EGI
DIRAC communities roundtable
LHCb:
Federico+Alexandre+Ryun+Vladimir+Alexey+Robin+Theau
- Running quite stable
Belle2
Dhiaraj
- NTR
CTAO
Natthan
- Finally have a running test setup v9 + DiracX (on Rancher) instance at DESY cluster.
- intending to connect to Rucio
- Natthan will try to write down the issues and the documentation once ready. Few changes to the charts done – will be documented!
GridPP:
Daniela, Simon, Janusz
- Updated production to v8.0.74, uneventful so far.
- When logging into the cvmfs uploader I noticed that we use a lot of space for releases that presumably will never be used: Can we e.g. delete v9.0.0a* up to the release LHCb is using ?
- ChrisB CVMFS should be de-duplicating things
- Simon this does not seem the way RAL has set up CVMFS
- ChrisB CVMFS should be de-duplicating things
- On the other hand, can we please add the latest v8.0 release ? At the moment they stop at 71. (There might be some v8 releases that could go as well.)
- Federico there is an issue with the uploader, @Andrei should look into it
Topics from GitHub discussions and bots
DIRAC releases
- v8.0.74
- Fixes for platform detection
- v9
- keep doing mostly fixes PRs and releases. Some pull requests with slighltly bigger changes in there, tbd
DIRAC projects
DIRAC:
Issues by milestone:
- v8.0:
- NTR
- v9.0:
- Empty
- After v9.0:
- Closing a few that won’t be tackled because of lack of time
- from previous meeting Asked about Is PilotBundle.bundleProxy() still useful?
- No – so, code removed with PR, and issue closed
- Replacement for BDII2CSAgent #8194
- The first and main issue is that GOCDB does not support non-WLCG VOs. A ticket (through GGUS Helpdesk) will be open
PRs discussed:
- refactor: move DIRAC WebApp related RPC calls to dedicated service #8190
- To ease the migration to DiracX. For Jobs, Pilots, TS (the other WebApp specific calls were already sandboxed)
- [9.0] JobDB snapshot: use a summary table for jobs in final state #8181
- ChrisB a MySQL trigger is the way to go … to be discussed offline
- [9.0] StalledJobAgent improvements #8192
- should be a simple one to review
WebApp:
- Fixes PR for v9 merged last week, other waiting (to go together with DIRAC#8190)
Pilot:
- a new version went into production: https://github.com/DIRACGrid/DIRAC/discussions/7512#discussioncomment-13118082
- Pilot migration
- diracx
pilotroute
- diracx
- from previous meeting PR feat: Adding JWT support alongside X509 auth
- the new Pilot command can call directly the route, no need to use the CLI
- the integration tests for this will be set up once diracx is updated with the connected diracx PR
- for this to be done neatly, the branching strategy of diracx will need to be defined
- there will be no branching of DiracX for the moment
- for this to be done neatly, the branching strategy of diracx will need to be defined
DIRACOS:
- NTR
Documentation:
- diracgrid.org being revamped. It will have:
- info about the consortium
- generic info, independent from DIRAC and DiracX
- links to DIRAC and DiracX documentation
OAuth2:
- NTR
management
- NTR
diraccfg
- NTR
DB12
- from previous meeting https://github.com/DIRACGrid/DIRAC/issues/7760#issuecomment-2482420604
- Federico proposed to create “alternate” benchmark
Rucio
Tests
- from previous meeting Federico Started adding Rucio to Dirac integration tests
- –> to Janusz
DiracX:
- Road Map : https://github.com/DIRACGrid/diracx/blob/caf66076bc8b623b0282a2f3d7d723b7c45be1a6/docs/roadmap.md?plain=1
- the only thing left to complete before v0.1 is documentation
- dependabot alert
- NTR
Issues
- Try to setup release-please (or alternatives … semantic-releases?) #264
- Alexandre studying semantic-releases. Seems alright, when ready it will be used for all diracx projects
- Breaking the API breaks the web interface #350
- the current decision is “not yet needed. Maybe never”
PRs discussed:
- Federico There are many “draft” PRs. Any chance of un-drafting few of them?
- Is fix: #448 and smarter datetimes #454 ready?
- Jorge looks OK to me but I can’t know all the implications
- ChrisB It is probably OK, but will try to understand it better
- Federico bonus question: if it’s ready and can be merged, does it mean that we can give it another go at postgres support?
- Jorge will try to revive his previous PR about postgres with this fix in, for testing purposes
DiracX-charts:
- NTR
DiracX-web:
- Theau implemented a way for share app states (dump in JSON, reload through web)
- the feeling is that it’s the first step, but a better way to share should be implemented. Maybe look e.g. what grafana does. The produced json can be easily compressed, maybe add to URL (the current way of constructing should disappear)
Documentation
- tasks: https://github.com/DIRACGrid/DIRACx/issues?q=is%3Aissue state%3Aopen label%3Adocumentation
- Created the documentation with mkdocs. Hosted on readthedocs.
- ChrisB registered https://diracx.io/ time ago, linking rtd docs
- skeleton quite alright, the content needs large revision
Release planning, tests and certification
-
Certification machines
- NTR for today
- Federico will update it with the latest diracx goodies
-
Next hackathon(s)
- We should go back at not testing in production (e.g. many changes for pilots from Robin)
-
Tag DIRAC v9 (and diracx, web, etc): ready!
-
Few DB changes being prepared (DIRAC):
- remove the TransformationDB.DataFiles table
- added foreign keys to PilotAgentsDB
- and we might be discovering more… 9.1?
Next appointments
-
Meetings:
- BiLD: in 3 weeks
-
WS/hackathons/conferences:
-
DIRAC Users’ Workshop
📅 17–20 September 2025 | 📍 IHEP, Beijing, China
🔗 indico.cern.ch/e/duw11
* Connect with the DIRAC community
* Meet the developers
* Discover DiracX
* Share your insights
The workshop will feature talks from users, administrators, and developers — plus a hands-on DiracX Hackathon.Poster in https://cernbox.cern.ch/remote.php/dav/public-files/1egpqkseV0GvqlO/DIRAC_Poster.pdf – make use of it!
-
AOB
- fromPreviousMeeting DIRAC was invited to be an “HSF affiliated project” : https://hepsoftwarefoundation.org/projects/affiliated.html
- we agreed (!)
LHCbDIRAC
- StorageReport app available in https://lhcbdiracx-cert.app.cern.ch/
- few minor issues still to be sorted out before accepting the PR
- fromPreviousMeeting Theau is having issues running everything (lhcbdiracx-web + lhcbdiracx) on his provided computer. Federico suggests to use a virtual machine, check with Ian if he can a large flavor in his personal tenent.
- 15th May better now
- fromPreviousMeeting Alexandre posted update to Moving
Job finalizationstep from the workflow to theJobWrapper: Transition Plan for Enhancing HPC Exploitation in DIRAC/LHCbDIRAC with connected draft PR Draft: feat(wms): New LHCb workflows- 15th May no news
There are minutes attached to this event.
Show them.
