BiLD-Dev

Thursday 20 Feb 2025, 10:00 → 12:00 Europe/Zurich

2/R-014 (CERN)

Federico Stagni (CERN)

Bi-Weekly "Loyal" DIRAC developers meeting. And, following, the LHCbDIRAC developers meeting.

Zoom: BiLD
https://cern.zoom.us/j/62504856418?pwd=TU1kb01SOFFpSDBJeWVBdU9qemVXQT09

Meeting ID: 62504856418
Passcode: 12345678
 

Notes

 

 

BiLD – 20/02/2025

At CERN: Federico, André, Christophe, Alexandre, Christopher, Ryunosuke
On Zoom: Andrei, Daniela, Simon, Janusz, Hideki, Ueda, Natthan, Henryk, Alexey, Xiaomei
Apologies: Jorge

---

Follow-up from previous meetings

• Last BiLD was Feb 6th
• Last DIRAC certification hackathon on Feb 13th
  • https://github.com/orgs/DIRACGrid/projects/23
  • With participation (apart from the usual suspects) from Henryk and Natthan
  • Pretty OK, but did not finish fully

---

DIRAC communities roundtable

LHCb:

Alexandre+Christophe+Christopher+Vladimir

• Using bulk submission (parametric jobs) for all production types (TransformationSystem WorkflowTaskAgent option), with large-ish speed-ups
• fromPreviousMeeting
  • Prepared a lhcbdiracx instance (with lhcbdiracx and lhcbdiracx-web), starting to run tests there
    • After going into production, we will release v9+0.1
  • Running a stress test (a “ping” from the pilots)

ILC/Calice/FCC

André

• NTR

EGI

Andrei

• Still looking for an engineer to hire for the GreenDIGIT project to work on the EGI DIRAC WMS.
• For GreenDIGIT, hoping to team up with LHCb’s Henryk on it.

Belle2

Hideki, Ueda

• Suffering of DB issue, bulk submission becomes difficult
• Paramers on OpenSearch: how to
  • Federico just enable it, the system will do the rest. After a while, check if there are still parameters on MySQL

GridPP:

Daniela, Simon, Janusz

• Nothing to report.

Juno

Xiaomei

• Asking about https://github.com/DIRACGrid/DIRAC/discussions/8042#discussioncomment-12230945
  • Christophe added suggestion in the discussion
  • Federico was suggesting to add a “withFlush” transformation plugin to the standard ones.

CTAO

Natthan

• Nothing to report
• fromPreviousMeeting
  • Some issues:
    • issue with dex IdP (can we run without it?)
    • issue not related to diracx but currently problem with nginx controller "path type error with /.well-known"
    • “Empty cartesian product” error in SiteDirector

---

Topics from GitHub discussions and bots

• only un-answered DIRAC and DiracX topics with discussion updates:
  • NTR

---

DIRAC releases

• v8.0.68
  • new command dirac-apptainer-exec
  • Moved reference MySQL version to 8.4
  • Parametric jobs should be OK for all job types

---

DIRAC projects

DIRAC:

Issues by milestone:

• v8.0:
  • NTR
• v9.0:
  • 2 issues added after last week’s hackathon
• After v9.0:
  • adding here those issues that will be treated in the lifetime of v9

Other issues:

• Several issues open since long, some might be solved already. Please review those that you opened yourself

PRs discussed:

• nothing interesting

WebApp:

• from previous meeting One draft PR

Pilot:

• Federico will add possibility to use dirac-apptainer-exec for running command in the pilot (mostly for dirac-platform)
• from previous meeting Janusz some doc to write
  • Started writing it!

DIRACOS:

• Release 2.50 : https://github.com/DIRACGrid/DIRACOS2/releases/tag/2.50 (upper constraint for davix – now marked as “fixed” in the referenced issue)

Documentation:

• Janusz issue with creating the documentation, discussion open

OAuth2:

• NTR

management

• Added an image for creating CA and certificates for integration tests: https://github.com/DIRACGrid/management/pull/65 (used in DIRAC’s draft PR, failing for now…)

diraccfg

• NTR

DB12

• from previous meeting https://github.com/DIRACGrid/DIRAC/issues/7760#issuecomment-2482420604
  • Federico proposed to create “alternate” benchmark

Rucio

• [8.0] Implementation of findfilesbymetadata 's method into rucio file catalog client
  • Cedric to review

Tests

• from previous meeting Federico Started adding Rucio to Dirac integration tests

---

DiracX:

• Road Map : https://github.com/chaen/diracx/blob/roadmap/docs/ROADMAP.MD

• dependabot alert

  • nothing that it is “solvable” by us

• Projects for ISIMA student:

  • Pilot (security model)
    • note
    • a bit lengthy discussion, mostly for general understanding. It is agreed that there should be a more detailed planning before the student starts. A few of us will be called in dedicated meeting in the next days.

Issues

• None discussed

PRs discussed:

• Federico thinks the reviews should be more prompt, and not only following on the shoulders of Chris&Chris
• fromPreviousMeeting OTel proof of concept PR: https://github.com/DIRACGrid/diracx/pull/379
  • This is a poc PR that is not yet merge-able. Christophe giving instructions to Jorge on how to proceed

DiracX-charts:

• NTR

DiracX-web:

• NTR

---

Release planning, tests and certification

• Certification machines

  • Updated MySQL to 8.4.2

• Next hackathon(s)

  • next Thursday

Next appointments

• Meetings:

  • hackathon: next week
  • BiLD: in 2 weeks

• WS/hackathons/conferences:

  • DiracX hackathon: 5 and 6 May - https://indico.cern.ch/event/1501369/
    • registrations open
  • Dirac Users’ Workshop: 17th-20th September 2205 - https://indico.cern.ch/e/duw11
    • time to start planning your trip! (you might need a VISA)

AOB

• DIRAC was invited to be an “HSF affiliated project” : https://hepsoftwarefoundation.org/projects/affiliated.html
  • We should answer after a consortium decision. It’s not clear if there are benefits at all, or even something extra required on our side
• DIRAC got an honorable mention (via EGI) during the inaugural meeting of the “Network for Sustainable Digital Research Infrastructure” meeting in the UK: NetDrive Meeting -GreenDIGIT

---

LHCbDIRAC

• LHCbDIRAC
  • Alexey restarted work on Move getFileDescendents to NewOracleBookkeepingDB
    • work on DB done, green light on creating related service/client API for it.
    • PR un-drafted, should be OK to merge
  • Sasha created issue Extending the bookkeeping with a CONDITIONAL DQ flag
    • generated PR
• LHCbDiracX
  • ISIMA for LHCb specific (LHCb extension, also for the Web): note
    • After a bit lengthy discussion we think that the student could create a brand new BookkeepingBrowser Web App in LHCbDiracX-Web. The thing do not (probably shouldn’t at all) mimic the existing Web App, rather it should be an opportunity for re-designing it completely. So, start with requirements and user stories.
  • certification setup: updates?
• Federico will update MySQL to 8.4.2 for the certification setup. I will follow up for the production databases

 

10:00 → 10:10 Items from Previous BiLD-Dev

10:10 → 10:20 DIRAC Communities roundtable

10:20 → 10:30 DIRAC releases

10:30 → 10:55 DIRAC projects

• DIRAC
• WebApp
• Pilot
• DIRACOS2
• VMDIRAC
• Documentation
• OAuth2
• DiracX
• other externals (include Rucio)

10:55 → 11:00 Release planning, tests and certification

11:00 → 11:15 Weekly development(s) focus

11:15 → 11:25 AOB

Convener: Federico Stagni (CERN)

11:25 → 11:40 LHCbDIRAC