BiLD (Bi-weekly DIRAC Development meeting) – 09/02/2023
On Zoom: Federico, Andrei, Alexey, Alexandre, Christophe, Christopher, Daniela, Domenico, Ewoud, Simon, Janusz, Xiaomei
Apologies: André
Follow-up from previous meetings
- Last BiLD 2 weeks ago
- Hackathon planned (but did not happen) last week
- “as usual” lately, the PRs were not ready.
DIRAC communities roundtable
LHCb:
Federico+Alexandre+Christophe+Christopher+Alexey
- Running v8.0 in production for 3 weeks
- from previous meeting Have been asked where we stand concerning token deployment
- need quick-ish what’s being developed
EGI
Andrei
- Not yet updated to v8.0, mostly DB updates needed first.
ILC/CLIC/FCC/Calice:
Belle2
- NTR
- from previous meeting Production to v7r2, but need to move to v8.0 before May, for HTCondor support, potentially also for Rucio support
GridPP:
Daniela+Simon+Janusz
- Daniela Tested v7.3.35 in preprod, no blockers; will put it in production if I get a quiet minute.
juno
Xiaomei
- Trying to move v8.0 in May, need to test out the token-based submission
Topics from GitHub/Discussions or Google forum
only un-answered topics:
DIRAC releases
- v7r3
- v7.3.33 and v7.3.34 and v7.3.35
- several fixes for ARC and AREX
- v7.3.35
- more fixes (mostly WMS)
- Adds support for OpenSSL 3.0.x
- Needs to create a new release though
- v8r0
- v8r1
- Haven’t done one in a while
DIRAC projects
DIRAC:
Issues by milestone:
- v7r3:
- 5 open issues, 3 of them with PRs ready to close them
- v8.0:
- 22 open issues, 2 of them recently created
- v8.1:
- 22 open issues
- Few updates below
Other issues:
- 2 new issues added for documentation fixes
PRs discussed:
- [8.0]Pilot submission with tokens
- Reviewed with good suggestions. Andrei Some simplifications can go in, we don’t need much abstractions as anyway we would stick to only IAM and CheckIn, and interacting to them is rather similar/simple.
- Somewhat urgent. Reminder to stick with what has been discussed in previous topic-specific meetings, specifically:
- “Groups” are internal to DIRAC, tokens will not have such a notion
- no
oidc-agent
needed nor want to
- [8.0] Pilot stamps available in the pilot execution environment
- Federico Some PRs of mine have been created long time ago and they are ready. Since no review is coming, I will merge them without review tomorrow, unless someone does one.
- Chris and Chris Reviewing PRs containing several style-related commits mixed with real changes is difficult
WebApp:
- Few PRs open, connected to DIRAC PRs
Pilot:
DIRACOS2:
- from previous meeting Chris Is anyone using v7r2 client with py3?
- Still, for a few more days (CLIC), a bit longer (May, Belle2)
- Chris we might end up breaking v7r2 support
- will need to update to OpenSSL 3 early next year as OpenSSL 1.1.1 will reach EOL soon: https://github.com/DIRACGrid/DIRACOS2/pull/95
- 9th Feb Seems ready to go
- MySQL 8.0.27 version forced, as it drops support for old TLS
- and VOs using MySQL 5.7.9 or earlier? That would break with the next DIRACOS release
- We could potentially use python 3.10, or even 3.11 already
Documentation:
OAuth2:
tornado/HTTPs
management
- from previous meeting 3 issues left, still valid
diraccfg
- from previous meeting Christopher Do we want to make a release that drops support for py2?
COMDIRAC
Daniela
DB12
Rucio
Tests
Release planning, tests and certification
Certification machines
- lbcertifdirac70 machine:
- Computing Element of each supported type should be configured (HTCondorCE, 2 ARCS (Alexandre (maybe use RALPP and Glasgow ? I can talk to them if there are issues --Daniela)), 1 Cloud (Daniela - this is already part of the tests), SSH). Consider OAuth tests as well - what is needed as far as certification servers configuration is concerned.
- Daniela: Enabled (and tested, at least with X509) WLCG VO for token testing. Needs update in puppet for /etc/grid-security/vomsdir and /etc/vomses, please.
- Christophe I’ll check and add it
- Daniela Still missing as of 14/12/22 A working config can be found at /cvmfs/grid.cern.ch/etc/grid-security/vomsdir/wlcg/ and /cvmfs/grid.cern.ch/etc/grid-security/vomses/wlcg-voms.cloud.cnaf.infn.it
- lbcertifdiracoauth machine:
Next hackathon(s)
- February 16th on lbcertifdirac70
AOB
Next hackathon: February 16th (Daniela will be away)
Next BiLD: February 23rd
LHCbDIRAC
- v11.0: deploy board in https://trello.com/b/Ep0PAkbv/deploy-110
- new dashboards added in Kibana, but issues for few of them. Issue created in DIRAC, followed up by Ewoud
- New DMS dashboards in https://monit-grafana.cern.ch/d/Q78h6E-nz/home?orgId=46
- Federico are these double checked? “safe” enough to be advertised?
- https://lhcb-auth.web.cern.ch/
- from previous meeting Andrei this is not properly configured yet. Not usable for Pilot submission yet (compute scopes not added).
- fixed? Alexandre Apparently no, seems like the
compute
scope does not exist (compute.cancel/create/modify/read
). Update: they don’t appear by default but they exist. - hackathon proposed this February? https://indico.cern.ch/event/1236132/ (it is today lol)
- seemingly not for us, though
- We should create a new production release as we are running with hotfixes. MRs seems to be all ready for merge.
- MR for the Boookeeping: https://gitlab.cern.ch/lhcb-dirac/LHCbDIRAC/-/merge_requests/1354
- add a proxy class for Bookkeeping (explanation in the MR itself)
- should be reviewed commit by commit
- will give the possibility to “safely” change the BookkeepingDB Oracle queries, that are otherwise rather painful to debug
- Alexey seems like the correct idea.
There are minutes attached to this event.
Show them.