Focus on:  Hide Contributions
Login

BiLD-Dev

→ Europe/Zurich
2/R-014 (CERN)

2/R-014

CERN

10
Show room on map
Federico Stagni (CERN)
Description

Bi-Weekly "Loyal" DIRAC developers meeting. And, following, the LHCbDIRAC developers meeting.

Zoom: BiLD
https://cern.zoom.us/j/62504856418?pwd=TU1kb01SOFFpSDBJeWVBdU9qemVXQT09

Meeting ID: 62504856418
Passcode: 12345678
 

working notes
BiLD
Zoom Meeting ID
62504856418
Host
Federico Stagni
Useful links
Join via phone
Zoom URL
Hide

BiLD (Bi-weekly DIRAC Development meeting) – 10/08/2023

At CERN: Christophe, Vladimir
On Zoom: Federico, Bertrand, Christopher, Simon F, Ueda, Janusz, Lorenzo, Kushagra Singhm, Ewoud, Xiaomei, Alexey, Simon M
Apologies: Daniela, Andrei

Follow-up from previous meetings

  • Last “standard” BiLD 2 weeks ago
    • Trying to catch all updates below
  • Last DIRAC certification hackathon on August 3rd

DIRAC communities roundtable

LHCb:

 Federico+Simon+Christophe+Christopher

  • Running on latest v8.0 and latest DIRACOS2 versions
  • Using tokens for submitting to almost all HTCondorCEs and some AREX
  • Latest v8 WebApp was problematic for LHCb, and had to remove the “Tabs” from cfg.

CLIC/ILC/Calice

 Lorenzo

  • NTR

Belle2

 Ueda, Kushagra

  • v7r3 py3 on servers – py3 for clients with some exceptions
  • pilot still uses py2 – for setting up DIRAC, not installing it
  • preparation for DIRAC v8 ongoing. Right now some issues while adapting WebApp.
  • from previous meeting DIRACOS2 migrated to py3.11, showed incompatibility with one of the external services.

Juno/BES3

 Xiaomei

  • Moving to v8.0
  • Ran into py3.11 issue with https://github.com/DIRACGrid/DIRAC/discussions/7165
    •  Christophe please put new error in the ticket
    •  Federico Seems more like an openssl error rather than python version
      • and ideally try to reproduce the error with pure openssl command
    • ticket solved later in the day

GridPP:

 Simon, Janusz

  • No updates to the production server. (No v8.0 relase with ARC-CE fixes yet ?)
  • Tested Chris H’s SingularityCE fixes on pre-prod, seems to work.
  • Still working on TransformationSystem for v8.0 for improved security model, waiting for input from Luisa wrt Production system: [#7113] (https://github.com/DIRACGrid/DIRAC/pull/7113)
  • For integration, we will port the code using username instead of userDN now that #7124 is resolved.
    • We could really do with a working certification server because otherwise we can’t test it, Daniela hasn’t managed to run a job on integration for months now, grumble.
  • No news on tokens (i.e. we haevn’t done any further work on this): #7123 (also: #7126)
  • Daniela & Simon have now booked their flights to Japan :-)

Topics from GitHub/Discussions

only un-answered topics with discussion updates:

DIRAC releases

  • v7r3
    • Latest patch incorporates all fixes
  • v8r0
  • v8r1
    • v8.1.0a16
      • included PilotLogging

DIRAC projects

DIRAC:

Issues by milestone:

v8.0:

v8.1:

Other issues:

PRs discussed:

WebApp:

  • 2 accompanying PRs

Pilot:

  •  Federico general suggestion for development on Pilot code in order to avoid “overconfident” merges of PRs to master branch, as experience has shown that it is often delicate. They would also make for a simpler workflow.:
    1. align master with devel (~now)
    2. forbid opening of PRs to master branch
    3. always test devel in DIRAC certification setup (and in github Actions and in Jenkins)
    4. (more) frequently merge devel to master
  • Discussion
    •  Daniela (in absentia): This only works if it is actually possible to run jobs in certification.
    •  Christopher if we have many changes in devel it will be difficult/risky to merge
      •  Federico I thought about having also another branch, e.g. releaseCandidate branch, but effectively we can always test in production also contents of PRs before merging them
    •  Chistopher can we think at a better way of testing this? Maybe submitting partly devel and partly master pilots.
    •  Federico points 1 and 2 above are probably OK anyway. We can wait for the next BiLD to see if there is any other suggestion
  • PRs:

DIRACOS:

  •  Christopher No longer installing Singularity and Apptainer, just Apptainer (with simlink to singularity)

Documentation:

Asking to Add documentation on how to write scripts that interact with DIRAC

mail from rtd:

We are removing the “use system packages” feature on August 29th. Make sure you are installing all the required dependecies to build your project’s documentation using a requirements.txt file and specifying it in your .readthedocs.yaml.

Here you have an example of the section required on the .readthedocs.yaml configuration file:

python:
install:
- requirements: docs/requirements.txt

  •  Christopher we’re using the Conda env, so nothing to do.

OAuth2:

  • from previous meeting
    •  Andrei request from EGI to demonstrate that one VO can run with tokens only
    • Check In is progressing: compute scopes available, they are accepting the idea of using client access tokens (possibility to associate a client to a given VO). They would probably not accept a same client to deal both with client and user access tokens (security concerns with the scopes available in the clients).
    • WLCG timeline document: https://zenodo.org/record/7014668#.YyLag9JBwQ9
  • Still pending the test for ARC7 and CheckIN tokens
  • Multiple clients per IdP might be effectively needed
  • Asked about multi-VO setup and token tags

management

  • from previous meeting 3 issues left, still valid?
  • Always upload releases to CVMFS is again valid for clients
    • higher priority now. This should also be done for non-x86 versions.

diraccfg

  • version 1.0? still tbd

DB12

 Alexandre

  • NTR

Rucio

  • NTR

Tests

  • NTR

DiracX

  • from previous meeting
    • Called a 2nd DiracX hackathon: https://indico.cern.ch/event/1304626/ for 4-5 September. Again at CERN, please do register if you plan to attend.
    • Also called a BildX meeting for August 31st (the week before the hackathon)

Release planning, tests and certification

Certification machines

  • lbcertifdirac70 machine:
    • NTR
  • from previous meeting  Federico not rush, but should we move to a Alma9 box?
    • Outside of CERN would be better, in CC probably
      •  Andrei machine is already there, need to decide how to set this one up
    • We could also use the new box to test the installation procedure

Next hackathon(s)

  • in 2 weeks, “standard” v8.1.0aX one.

AOB

Next hackathon: August 24th
Next BiLD: September 14th (no possibility before)

LHCbDIRAC

v11.0.17 and v11.0.18

  • Created v11.0.17 a bit in a hurry (DPA pushing) but changes for conditions_id impacted other parts. Decided to revert the specific PRs and create a new release.

v11.0: deploy board in https://trello.com/b/Ep0PAkbv/deploy-110

  • NTR

LHCbDIRAC hackathon?

  • 17th Aug. Will be mostly for restoring the setup and verify if there’s anything dramatically broken.
There are minutes attached to this event. Show them.
    • 10:00 → 10:10
      Items from Previous BiLD-Dev 10m
    • 10:10 → 10:20
      DIRAC Communities roundtable 10m
    • 10:20 → 10:30
      DIRAC releases 10m
    • 10:30 → 10:55
      DIRAC projects 25m
      • DIRAC
      • WebApp
      • Pilot
      • DIRACOS2
      • VMDIRAC
      • Documentation
      • OAuth2
      • DiracX
      • other externals (include Rucio)
      DIRAC certification
    • 10:55 → 11:00
      Release planning, tests and certification 5m
      DIRAC PRs
    • 11:00 → 11:15
      Weekly development(s) focus 15m
    • 11:15 → 11:25
      AOB
      Convener: Federico Stagni (CERN)
    • 11:25 → 11:40
      LHCbDIRAC 15m
Powered by Indico v3.3.7-pre