BiLDx

Thursday 16 Nov 2023, 10:00 → 12:00 Europe/Zurich

2/R-014 (CERN)

diracx developments meeting

 

Zoom: BiLD
https://cern.zoom.us/j/62504856418?pwd=TU1kb01SOFFpSDBJeWVBdU9qemVXQT09

Meeting ID: 62504856418
Passcode: 12345678

working notes

BiLDX – 16/11/2023

At CERN: Federico, Alexandre
On Zoom: Andrei, Christophe, Christopher, Cedric, Janusz, Andrei, Lorenzo, Simon F, Xiaomei, Vladimir, Janusz, Natthan
Apologies:

Follow-up from previous meetings

• Last BildX 3 weeks ago (Oct 26th) https://indico.cern.ch/event/1331295/
  • notes in https://demo.hedgedoc.org/H0DTIowtSRaFj810TjlRIQ
• Last DIRAC Hackathon: diracx issues, basically DiracX dependencies could not be satisfied. Will be fixed before the next DIRAC hackathon (which will be in 2 weeks)

Progress updates

• from previous meeting DiracX is now installed on CERN openshift against DIRAC certification
  • https://diracx-cert.app.cern.ch/
    • Andrei and Federico added to GridPP IAM
• from previous meeting Created a project board on GitHub for planning the first release

Topics for discussion

Release planning

• PR 173
  • This PR is splitting diracx in different packages. The documentation for this is being made in PR 172, which is where main comments should go
    • This was already advertised in https://indico.cern.ch/event/1321740/attachments/2707272/4700173/DiracX BiLD - Part 2.pdf from slide 40, even though a few things changed since then
    • Effectively, there will be 3 installable packages:
      • service
      • client
      • tasks
    • This namespaces packages are not too common but are an existing and documented feature of python
    • advantages: we can split the dependencies better. E.g. fastAPI will only be needed by the diracx-services package.
    • downsides:
      • requires a “dirty” directory structure
      • multiple pip - installable packages will be created, and a tag of the whole diracx repo will trigger the creation for all of them – there might be no difference between some of the released versions
    • diracx is still using conda/mamba for packaging its dependencies. We will then have different environment.yaml files for each of the created packages.
      • such conda environments will be used also for the docker containers
      • a “merge” of all these environment.yaml files will be “equivalent” to DIRACOS2.
  • The current DIRAC pre-release has a hard dependency on the completion of this PR, and then at a minimum diracx-core will need to be added to DIRAC dependencies.

GitHub discussions

•  Alexandre created Transitioning from (Dirac Worklfow, JDL) to (CWL, pydantic JobModel)
  • Plan seems OK, at least there’s nothing against the first step

from previous meeting Please give feedback on these by the next BiLDx (or forever hold your peace):

• Who uses request proxies?
• Does the web need to support anonymous access?
• Login page design for the new web app

Next steps

• Next BildX: Dec 7th
• Next DiracX hackathon: https://indico.cern.ch/event/1343151/
  • do register if you want to participate

AOB

• We will formalize the date of the DIRAC Workshop by next BiLD in a week

10:00 → 10:20 diracx

Speakers: Chris Burr (CERN), Christophe Haen (CERN)

10:20 → 10:50 DiracX: Github Discussions, Issues and PRs