Choose timezone
Your profile timezone:
BiLD-Dev
Description
Bi-Weekly "Loyal" DIRAC developers meeting. And, following, the LHCbDIRAC developers meeting.
Zoom: BiLD
https://cern.zoom.us/j/62504856418?pwd=TU1kb01SOFFpSDBJeWVBdU9qemVXQT09
Meeting ID: 62504856418
Passcode: 12345678
BiLD
BiLD (Bi-weekly DIRAC Development meeting) – 14/09/2023
At CERN: Federico, Vladimir, Alexandre
On Zoom: Bertrand, Ueda, Janusz, Xiaomei, Alexey, Simon M, Daniela, Simon F, Christophe, Christopher
Apologies: Andre, Lorenzo, Andrei
Follow-up from previous meetings
- Last “standard” BiLD on August 10
- Trying to catch all updates below
- Last DIRAC certification hackathon on 24/08 (https://trello.com/b/SpdezYWX/v810a18)
- on release v8.1.0a18
- Daniela: The usefulness of the last two hackathons was severely hampered by the fact that jobs didn’t run. Is it possible to make this fundamental step work before calling the next hackathon ?
- Federico yes, I will
- BildX on 31/08
- Discussion mostly around CWL
- Unfortunately Federico + Chris^2 could not attend
- DiracX hackathon on 4th/5th September: https://indico.cern.ch/event/1304626/
- 10 developers attended (new: Luisa+Natthan).
- Not many discussions, mostly “head down and code”. Progress in many aspects though.
- Realizing one by one all fixes that needs to be done to DIRAC first (not only refactoring).
DIRAC communities roundtable
LHCb:
Federico+Simon+Christophe+Christopher+Alexey
- Running on latest DIRAC v8.0 patch
Belle2
Ueda
- Q: would it be possible to run v8.0 on older (py3.9) DIRACOS?
- Federico+Christophe probably, but at your own risk. You should always run the latest.
- N New TR
- from previous meeting DIRACOS2 migrated to py3.11, showed incompatibility with one of the external services.
- openssl handshake failure – will open an issue, maybe there’s a workaround
- Federico Maybe same as https://github.com/DIRACGrid/DIRAC/discussions/7165
Juno/BES3
Xiaomei
- Running DIRAC v8.0.x
- Going to test in production running MultiCore jobs
- Federico works well in LHCb production, doc is there and decent.
GridPP:
Simon, Janusz, Daniela
- Nothing to report. Testing v8.0.28 on pre-prod.
- set up tokens?: #7123
- should be answered…?
ILC/Calice/FCC
- Regarding VOMS-Admin at CERN (for FCC VO), was told that “VOMS-Admin will most likely not be made available for EL9 and there some VOs that would need to move to the IAM service in the next months (deadline July 2024). Will track progress monthly.”
- Same for LHCb VOMS-Admin? Is DIRAC ready for that?
- Answer: most probably it is true for LHCb also, but we do not have a real answer yet.
Topics from GitHub/Discussions
only un-answered topics with discussion updates:
- NTR
DIRAC releases
- v7r3
- NTR
- v8r0
- v8r1
- v8.1.0a1x
*
- v8.1.0a1x
DIRAC projects
DIRAC:
Issues by milestone:
v8.0:
- 15+ open issues
- no one git discussed. It’s possible to close some of them.
v8.1:
- 15+ open issues
- Merged PR https://github.com/DIRACGrid/DIRAC/pull/7143
- Created connected issue Remove setup from Operations section of CS
- Q: how to handle this migration?
- With 2 updates in CS (first is a copy, second a removal)
- this one seems preferred.
- Backporting
Operationshelper changes to v8.0 and just “move”
- With 2 updates in CS (first is a copy, second a removal)
- Q: how to handle this migration?
- Connected: remove Setup(s) from
Systemsection of the CS. PR https://github.com/DIRACGrid/DIRAC/pull/7186 creates but not enough
- Created connected issue Remove setup from Operations section of CS
- Q: would anyone be against in removing ingestion of JobParameters in MySQL table? (reading would stay) (also requested in https://github.com/DIRACGrid/diracx/issues/29)
- This would be a simplification for
DiracX - Would make effectively the use of ES/OpenSearch for JobParameters mandatory
- Connected: https://github.com/DIRACGrid/DIRAC/issues/7191
- will stay until next BiLD
- This would be a simplification for
- Something from https://github.com/DIRACGrid/diracx/pull/64 ?
- OK for adding VO to DB
- Merged PR https://github.com/DIRACGrid/DIRAC/pull/7143
PRs discussed:
WebApp:
- NTR
Pilot:
- from previous meeting (need answer today)
- Federico general suggestion for development on Pilot code in order to avoid “overconfident” merges of PRs to
masterbranch, as experience has shown that it is often delicate. They would also make for a simpler workflow.:- align
masterwithdevel(~now) - forbid opening of PRs to
masterbranch - always test
develin DIRAC certification setup (and in github Actions and in Jenkins) - (more) frequently merge
develtomaster
- align
- Discussion
- Daniela (in absentia): This only works if it is actually possible to run jobs in certification.
- Christopher if we have many changes in
develit will be difficult/risky to merge- Federico I thought about having also another branch, e.g.
releaseCandidatebranch, but effectively we can always test in production also contents of PRs before merging them
- Federico I thought about having also another branch, e.g.
- Chistopher can we think at a better way of testing this? Maybe submitting partly
develand partlymasterpilots. - Federico points 1 and 2 above are probably OK anyway. We can wait for the next BiLD to see if there is any other suggestion
- ===> OK for the plan, can add a new
releaseCandidatebranch at any time
- Federico general suggestion for development on Pilot code in order to avoid “overconfident” merges of PRs to
- PRs:
DIRACOS:
- NTR
Documentation:
- NTR
OAuth2:
- from previous meeting
- Andrei request from EGI to demonstrate that one VO can run with tokens only
- Check In is progressing:
computescopes available, they are accepting the idea of using client access tokens (possibility to associate a client to a given VO). They would probably not accept a same client to deal both with client and user access tokens (security concerns with the scopes available in the clients). - WLCG timeline document: https://zenodo.org/record/7014668#.YyLag9JBwQ9
- Still pending the test for ARC7 and CheckIN tokens
- Asked about multi-VO setup and token tags
management
- from previous meeting 3 issues left, still valid?
- Always upload releases to CVMFS is again valid for clients
- higher priority now. This should also be done for non-x86 versions.
diraccfg
- version 1.0? still tbd
DB12
Alexandre
- NTR
Rucio
- NTR
Tests
- PR merged for
diracXtesting
DiracX
- Moving to regular BildX meetings
Release planning, tests and certification
Certification machines
- lbcertifdirac70 machine:
- NTR
- from previous meeting Federico not rush, but should we move to a Alma9 box?
- Outside of CERN would be better, in CC probably
- Andrei machine is already there, need to decide how to set this one up
- We could also use the new box to test the installation procedure
- Outside of CERN would be better, in CC probably
Next hackathon(s)
- next week, “standard” v8.1.0aX one.
AOB
Next hackathon: September 21st
Next BiLDX: September 28th
Next BiLD: October 5th
This will be the new structure (3 meetings rotation) for a while
- Workshop approaching quickly.
- Timetable not fully complete, might change a bit. Your input is appreciated.
- Federico on contributions:
- We will send few instructions for the communities reports
- I will call a meeting with few of you for agreements on the technical contributions
LHCbDIRAC
- v11.0: deploy board in https://trello.com/b/Ep0PAkbv/deploy-110
- NTR
- Hackathon on 27th
- Federico will create the release before
- Production release yesterday had 2 issues:
- PR YAML tested ended up being broken
- CB created MR and hotfixed CVMFS
- TransformationSystem can not be queried anymore by users. Simon F already created PR with fix
- PR YAML tested ended up being broken
- BKK MR https://gitlab.cern.ch/lhcb-dirac/LHCbDIRAC/-/merge_requests/1439/diffs created for fixing https://gitlab.cern.ch/lhcb-dirac/LHCbDIRAC/-/issues/15
- had to test by connecting to production BKK DB
There are minutes attached to this event.
Show them.