BiLD-Dev

Thursday 2 Nov 2023, 10:00 → 12:00 Europe/Zurich

2/R-014 (CERN)

Federico Stagni (CERN)

Bi-Weekly "Loyal" DIRAC developers meeting. And, following, the LHCbDIRAC developers meeting.

Zoom: BiLD
https://cern.zoom.us/j/62504856418?pwd=TU1kb01SOFFpSDBJeWVBdU9qemVXQT09

Meeting ID: 62504856418
Passcode: 12345678
 

working notes

BiLD (Bi-weekly DIRAC Development meeting) – 02/11/2023

At CERN: Federico, Cedric, Andre, Christophe, Christopher
On Zoom: Hideki, Ueda, Janusz, Xiaomei, Daniela, Simon, Lorenzo, Andrei, Igor, Vladimir
Apologies:

Follow-up from previous meetings

• Last BiLD on October 5th
  • Trying to catch all updates below
• Last DIRAC certification hackathon on October 12th on v8.1.0a23
  • https://trello.com/b/IM7LFevo/v810a23
  • Some DiracX bit added there. In general, went rather well.
• DIRAC+Rucio WS in KEK (October 16th-20th)
  • impressions? Comments?
• short BildX on October 26th

DIRAC communities roundtable

LHCb:

 Federico+Christophe+Christopher+Alexey

• Running on (almost) latest DIRAC v8.0 patch

Belle2

 Ueda+Hideki+cedric

• running v7.3
• started Pilot migration to Py3
• started testing v8. Some issues with WebApp (related to extension)

Juno/BES3/CEPC

 Xiaomei

• Wants to move to Pilot that allows installation from CVMFS
  •  Federico can use devel branch of Pilot
  •  Xiaomei will wait that it goes to master

ILC/Calice/FCC

 Andre, Lorenzo

• NTR

EGI

 Andrei

• Jobs submitted to ARC7 (using Check-In tokens)

JINR

 Igor

• Changed the Pilot, using devel in production. CVMFS updates slow
• Tried to use new approach for running on OpenNebula. Looks promising, still few issues to sort out.

GridPP:

 Simon, Janusz, Daniela

• No changes to Production (8.0.28), pre-prod on 8.0.30, but no chance for token testing yet

Topics from GitHub/Discussions

only un-answered topics with discussion updates:

• NTR

DIRAC releases

• v8r0
  • v8.0.31
    • NEW: (#7248) Add support for loading external providers in CloudCE
    • NEW: (#7248) Add OpenNebula6 (XMLRPC) provider for CloudCE/libcloud
    • NEW: (#7223) TransformationCleaningAgent: add CleanWithRMS option to clean files via the RequestManagementSystem asynchronously rather than serially in the agent run itself
    • CHANGE: (#7208) pilot submission with tokens in a multi-VO context
    • NEW: (#7176) add documentation about setting up DIRAC to submit pilots with tokens
• v9r0
  • “v8.1” becomes “v9.0”

DIRAC projects

DIRAC:

Issues by milestone:

v8.0:

• 15+ open issues
  • usual reminder for closing/moving old ones

v9.0:

• 15+ open issues
  •  Federico proposes to close Integration with RCAuth: done

Other issues:

• https://github.com/DIRACGrid/DIRAC/issues/7260
  •  Chris will take care

PRs discussed:

• Went through the PRs, nothing specifically discussed

WebApp:

• NTR

Pilot:

• MERGED PR Fix for CVMFS locations lookup
  • for devel. Many integration tests added.
  • Tested to be compatible with LHCbPilot extension
  • Running in production in JINR DIRAC installation
  •  Federico During workshop, sat down with Hideki, for Belle2, but using frozen old version
    • provided test to run
•  Federico It’s time to merge devel branch into master. And then forbid opening of PRs to master.
  • will be done after next week’s hackathon if no issues will surface.
• Anyone’s free to add own test(s).
• (draft) PR https://github.com/DIRACGrid/Pilot/pull/218 done for addressing issue https://github.com/DIRACGrid/Pilot/issues/219
  • But discussion on how to do things are needed
  •  Chris+Chris can add to POST
•  Andrei Some issues with CVMFS Stratum-0 at RAL, so updates not working for some weeks. Version of DIRAC repo old
  •  Chris RAL had a hardware failure
  •  Andrei still not fully working, should make a ticket

DIRACOS:

• NTR

Documentation:

• “createClient” is no longer working (for the documentation)

DiracX

• Okay to merge security model?
  •  Federico I guess so

OAuth2:

• NTR

management

• from previous meeting 3 issues left, still valid?
• Always upload releases to CVMFS is again valid for clients
  • higher priority now. This should also be done for non-x86 versions.

diraccfg

• NTR

DB12

 Alexandre

• NTR

Rucio

• Metadata method in RucioCatalog –  Cedric PR incoming

Tests

• DiracX tests
• Pilot tests

Release planning, tests and certification

Certification machines

• lbcertifdirac70 machine:
  • NTR

Next hackathon(s)

• next week
•  Federico would anyone be against in moving from Slack to (CERN) Mattermost?
  •  Daniela “Fine by me, I never look at either.”

AOB

Next hackathon: November 9th
Next BiLDX: November 16th
Next BiLD: November 23rd

DIRAC+X hackathon:

• https://indico.cern.ch/event/1343151/
• 23rd-24th Jan 2024
• CERN, “usual” room
• You need to register
• Anyone’s welcome to suggest specific topics
•  Federico attempts to upgrade to v9 pre-versions can be done

Next DIRAC Users’ workshop: originally planned for 27th-29th May 2024 (effectively 2 half and one full day) in Lyon https://indico.cern.ch/event/1341205, but issue with dates:

• Wednesday 1st is holiday (so not that week)
• Thursday 9th will be ascension so that week is potentially problematic (Wed also prob holiday in France)
• Third week (of the 13th) should be OK
• Monday 20th is probably holiday
• Last week (of 27th) The 27th is a holiday in the UK, plus it’s the school holidays.
• First week of June is LHCb week and Belle2 week

---

LHCbDIRAC

• v11.0: deploy board in https://trello.com/b/Ep0PAkbv/deploy-110
  • NTR

10:00 → 10:10 Items from Previous BiLD-Dev

10:10 → 10:20 DIRAC Communities roundtable

10:20 → 10:30 DIRAC releases

10:30 → 10:55 DIRAC projects

• DIRAC
• WebApp
• Pilot
• DIRACOS2
• VMDIRAC
• Documentation
• OAuth2
• DiracX
• other externals (include Rucio)

DIRAC certification

• Certification setups
• Cert task board teamplate
• Slack channel

10:55 → 11:00 Release planning, tests and certification

DIRAC PRs

11:00 → 11:15 Weekly development(s) focus

11:15 → 11:25 AOB

Convener: Federico Stagni (CERN)

11:25 → 11:40 LHCbDIRAC