BiLD-Dev

Thursday 12 Sept 2024, 10:00 → 12:00 Europe/Zurich

2/R-014 (CERN)

Federico Stagni (CERN)

Bi-Weekly "Loyal" DIRAC developers meeting. And, following, the LHCbDIRAC developers meeting.

Zoom: BiLD
https://cern.zoom.us/j/62504856418?pwd=TU1kb01SOFFpSDBJeWVBdU9qemVXQT09

Meeting ID: 62504856418
Passcode: 12345678
 

working notes

BiLD – 12/09/2024

At CERN: Federico, André, Christopher, Ryunosuke, Alexandre
On Zoom: Janusz, Simon, Xiaomei, Cedric, Alexey, Daniela, Andrei, Ueda, Hideki
Apologies: Christophe

hedgedoc is not anymore reliable for these notes!

• All old notes are archived in https://demo-archive.hedgedoc.org/ (for another 5 months) but the new https://demo.hedgedoc.org/ is cleaned up ~daily (at least once every 24h).
• All old BiLD notes are also in Indico Agendas.
• Using codimd.web.cern.ch from now on

Follow-up from previous meetings

• Last BiLD on August 15th
• Last DIRAC certification hackathon on June 6th
  • last on lbcertifdirac70
  • then attempted to set up new infrastructure, without success

DIRAC communities roundtable

LHCb:

 Federico+Alexandre+Christopher+Alexey+Ryunosuke

• “Business as usual”

ILC/Calice/FCC

 André

• Tweaking VOMS2CS / IAM Service interface to prevent _1 users: https://github.com/DIRACGrid/DIRAC/pull/7784/
• Also needed SCIM config in IAM service cf. https://github.com/indigo-iam/iam/issues/763 to be able to read nickname attribute
  • Thanks to @Chris H

IHEP

 Xiaomei

• NTR

Belle2

 Hideki, Cedric

• Migrating to EL9 (not in prod yet)
• Can we use the TS to set Tags on its job? – open discussion

GridPP:

 Daniela, Simon

• Nothing to report for production, we are still running 8.0.38 with one hot fix, I believe. No complaints.

Topics from GitHub/Discussions

only un-answered topics with discussion updates:

• Usage of diracproject.web.cern.ch
  •  Federico no answers. OK if we ditch it?
• getProxyinfo to get group properties
  • André was answering…

DIRAC releases

• v8r0
  • 8.0.51
    • NEW: (#7740) FileCatalogHandler: add function export_getFileDetails to get the (user) metadata for a list of LFNs

DIRAC projects

DIRAC:

Issues by milestone:

• v8.0:
  • Little to no movement (at least for “old” tickets)
• v9.0:
  • …same

Other issues:

• growing list of “notes” rather than issues.
  • Incorrect error strings reported for end-user application failures
    •  Federico Simon do you have a proposal?
  • Is it possible to add a last chance option for file transfers via FTS ?
  • Pilot status for pilots in PollTime related sleep cycle
    • interesting discussion, feel free to contribute

PRs discussed:

• [9.0] remove JobDB’s site mask
  • PR is ready. Long-awaited Simplification of JobDB ahead of moving to DiracX
  • this makes RSS “mandatory”. Which installation is not using it yet?
• [v9.0] feat: add a new job management mechanism in the PushJobAgent
  • ? “last” big changes for job submission
  • the driver is the PushJobAgent but the (old) logic is pretty much streamlined

DiracX:

• Juggled back and forth with typer update that was breaking our tests
  • turned out to be our fault: https://github.com/DIRACGrid/diracx/pull/290
• Fixes looking around
  • mind merging? (as of now there’s a conflict)
• feat: add SQL transaction model tests
• Implemetation of PilotAgentsDB schema:
  • https://github.com/DIRACGrid/diracx/pull/292
  • https://github.com/DIRACGrid/diracx-charts/pull/112

WebApp:

• NTR

Pilot:

• from previous meeting  Janusz some doc to write

DIRACOS:

• Installation errors for diracos
• from previous meeting Made a new release with libxml2 downgraded. Issue opened to gfal2 for proper fix
  • FTS/gfal developers are going to look at this now
  • ==> a new release is there, pending in conda-forge
• from previous meeting Requests for Apple-M2 compatibility
  • also asked to update doc…

Documentation:

• from previous meeting Need to decide on strategy for DiracX documentation –  André to take care?

OAuth2:

• NTR

management

• from previous meeting Always upload releases to CVMFS
  • still not working (did not work for v8.0.50, nor for 8.0.51
  •  Andrei created a new script, so PR needed

diraccfg

• NTR

DB12

 Alexandre

• NTR

Rucio

• NTR

Tests

• NTR

Release planning, tests and certification

• Certification machines

  •  Federico the support level for running services at in2p3 is not sufficient. I will move to k3s at CERN for everything in order to restart things
    • I would run all DBs, including MySQL, through k3s. This is of course not ideal for a production environment, but for the test setup is hopefully OK
    • 3 VMs for DiracX
    • 1 VM for DIRAC v9

• Next hackathon(s)

  • in 2 weeks (next week is the DiracX hackathon)
    •  Federico I will do my best to have it ready

Next appointments

• Meetings:

  • hackathon: in 2 weeks
  • BiLD: in 3 weeks

• WS/hackathons/conferences:

  • DiracX hackathon: 16th-17th September, CERN: https://indico.cern.ch/event/1431888/ – do register!
  • CHEP 2024 19th-25th Oct, Krakow
    • 1 talk in plenary (Federico)
    • 2 talks in parallel (Natthan, Xiaowei)
    • 1 poster (Alexandre)
  • Dirac&Rucio mini-workshop and hackathon: 16th-17th January 2025, CERN https://indico.cern.ch/event/1443765/
    • Almost certainly we will have a DiracX hackathon on 14th-15th January
  • Dirac Users’ Workshop: 17th-20th September 2025 - https://indico.cern.ch/e/duw11

AOB

• INDIGO IAM Technical Hackathon - November 2024
  • any interest?
    •  Daniela will probably join

---

LHCbDIRAC

• BKK: answer to the ticket not satisfactory
• Some runs without luminosity. Total luminosity is there.
  • Frèderic Hemmer to check it
• from previous meeting BKK: The getDescendants implementation got faster, but does not have tree-like implementation, so that will need to be done.  Chris to answer

10:00 → 10:10 Items from Previous BiLD-Dev

10:10 → 10:20 DIRAC Communities roundtable

10:20 → 10:30 DIRAC releases

10:30 → 10:55 DIRAC projects

• DIRAC
• WebApp
• Pilot
• DIRACOS2
• VMDIRAC
• Documentation
• OAuth2
• DiracX
• other externals (include Rucio)

10:55 → 11:00 Release planning, tests and certification

DIRAC PRs

11:00 → 11:15 Weekly development(s) focus

11:15 → 11:25 AOB

Convener: Federico Stagni (CERN)

11:25 → 11:40 LHCbDIRAC