BiLD-Dev

Thursday 15 Aug 2024, 10:00 → 12:00 Europe/Zurich

2/R-014 (CERN)

Federico Stagni (CERN)

Bi-Weekly "Loyal" DIRAC developers meeting. And, following, the LHCbDIRAC developers meeting.

Zoom: BiLD
https://cern.zoom.us/j/62504856418?pwd=TU1kb01SOFFpSDBJeWVBdU9qemVXQT09

Meeting ID: 62504856418
Passcode: 12345678
 

working notes

BiLD – 15/08/2024

At CERN: Federico, André, Christophe, Christopher, Loris
On Zoom: Janusz, Simon, Xiaomei, Cedric, Alexey, Daniela (slightly late, mentally not quite back from holiday)
Apologies:

hedgedoc is not anymore reliable for these notes!

• All old notes are archived in https://demo-archive.hedgedoc.org/ (for another 5 months) but the new https://demo.hedgedoc.org/ is cleaned up ~daily (at least once every 24h).
• All old BiLD notes are also in Indico Agendas.
• Using codimd.web.cern.ch from now on… (at least for this one)
  • who does not have access?

Follow-up from previous meetings

• Last BiLD on July 4th
• Last DIRAC certification hackathon on June 6th
  • last on lbcertifdirac70
  • then attempted to set up new infrastructure, without success

DIRAC communities roundtable

LHCb:

 Federico+Alexandre+Christophe+Christopher+Alexey+Loris

• NTR

ILC/Calice/FCC

 André

• CALICE IAM now also at CERN

IHEP

 Xiaomei

• Asked about staging procedures and best practices

Belle2

 Cedric

• Moving to Alma9

GridPP:

 Daniela, Simon, Janusz

• NTR

Topics from GitHub/Discussions

only un-answered topics with discussion updates:

• M2Crypto status
• Usage of diracproject.web.cern.ch

DIRAC releases

• v8r0
  • 8.0.50
    • NEW: (#7739) Configuration-system-shell: added reload and sort commands
    • FIX: (#7690) Proper killing of jobs when not matched, running or stalled
    • FIX: (#7725) Update CloudCE cloudinit.template for EL8+

DIRAC projects

DIRAC:

Issues by milestone:

• v8.0:
  • Little to no movement (at least for “old” tickets)
• v9.0:
  • Elastic JobParameters improvements: PR created (before summer) solved most of them

Other issues:

• Remove DataFiles table from TransformationDB
• Database optimizations
  • Something done?

PRs discussed:

• [9.0] jobParametersDB improvements
• [9.0] Added index template for ElasticJobParametersDB
• [9.0] add ElasticPilotParametersDB
  •  Federico implement only in DiracX?

DiracX:

• from previous meeting  Loris DiracX-Web extensions
  • Slides presented (added to the agenda)

WebApp:

• NTR

Pilot:

• from previous meeting  Janusz some doc to write

DIRACOS:

• Installation errors for diracos
• from previous meeting Made a new release with libxml2 downgraded. Issue opened to gfal2 for proper fix
  • FTS/gfal developers are going to look at this now
  • ==> a new release is there, pending in conda-forge
• from previous meeting Requests for Apple-M2 compatibility
  • also asked to update doc…

Documentation:

•  André: Enabled “Read The Docs” Add-on (to be default soon anyway) to test if anything breaks – https://dirac.readthedocs.io/en/latest/
  • Floating Menu is now in a different spot
• from previous meeting Need to decide on strategy for DiracX documentation –  André to take care?

OAuth2:

• NTR

management

• from previous meeting Always upload releases to CVMFS
  • still not always working (did not work for v8.0.50)
  •  Andrei created a new script, so PR needed

diraccfg

• NTR

DB12

 Alexandre

• NTR

Rucio

• Dirac&Rucio mini-workshop and hackathon: https://indico.cern.ch/event/1443765/
  • 16th - 17th January 2025 at CERN

Tests

• PR mount cvmfs in pilot container

Release planning, tests and certification

• Certification machines

  • DIRAC: Alma9 VM ccdirac12.in2p3.fr
    •  Federico made some progress before holidays, now waiting for DiracX to be installed
      • Also running at least OpenSearch on DiracX cluster
  • DiracX: Alma9 VMs: ccdiracx01.in2p3 ccdiracx02.in2p3
    •  Federico installed k3s following the documentation provided at workshop
      • now waiting for some networking fixes…

• Next hackathon(s)

  • next week, if machines will be ready
    •  Federico which I doubt

Next appointments

• Meetings:

  • hackathon: in 1 week (if the machines will be ready)
  • BiLD: On August 29th ( Federico I might replace this with a hackathon)

• WS/hackathons/conferences:

  • DiracX hackathon: 16th-17th September, CERN: https://indico.cern.ch/event/1431888/ – do register!
  • CHEP 2024 19th-25th Oct, Krakow
    • 1 talk in plenary (Federico)
    • 2 talks in parallel (Natthan, Xiaowei)
    • 1 poster (Alexandre)
  • Dirac&Rucio mini-workshop and hackathon: 16th-17th January 2025, CERN https://indico.cern.ch/event/1443765/
    • Almost certainly we will have a DiracX hackathon on 14th-15th January
  • Dirac Users’ Workshop: 17th-20th September 2025 - https://indico.cern.ch/e/duw11
    • starting one day earlier than planned

AOB

•  

---

LHCbDIRAC

• from previous meeting BKK: The getDescendants implementation got faster, but does not have tree-like implementation, so that will need to be done.  Raja will test it soon as it is there.

10:00 → 10:10 Items from Previous BiLD-Dev

10:10 → 10:20 DIRAC Communities roundtable

10:20 → 10:30 DIRAC releases

10:30 → 10:55 DIRAC projects

• DIRAC
• WebApp
• Pilot
• DIRACOS2
• VMDIRAC
• Documentation
• OAuth2
• DiracX
• other externals (include Rucio)

10:55 → 11:00 Release planning, tests and certification

DIRAC PRs

11:00 → 11:15 Weekly development(s) focus

11:15 → 11:25 AOB

Convener: Federico Stagni (CERN)

11:25 → 11:40 LHCbDIRAC