BiLD (Bi-weekly DIRAC Development meeting) – 23/02/2023
On Zoom: Federico, Andrei, André, Alexey, Alexandre, Cedric, Christophe, Christopher, Daniela, Ewoud, Lorenzo, Simon, Janusz
Apologies: Ueda, Hideki
Follow-up from previous meetings
DIRAC communities roundtable
LHCb:
Federico+Alexandre+Christophe+Christopher+Alexey
EGI
Andrei
- Contemplating how to migrate to 8.0, using a test server especially for the DB updates
- running v7.3/py3, few services on 8.0
ILC/CLIC/FCC/Calice:
André, Lorenzo
Belle2
Cedric
- Trying v7r3.
- from previous meeting Production to v7r2, but need to move to v8.0 before May, for HTCondor support, potentially also for Rucio support
GridPP:
Daniela+Simon+Janusz
- Lots of issues with AREX and proxy renewals (classic small VO side effects: our jobs tend to queue a lot longer than LHCb’s) - Simon submitted a bunch of patches. Still on v7.3.34 + hotfixes (see issues wrt v7.3.35)
Topics from GitHub/Discussions or Google forum
only un-answered topics:
DIRAC releases
- v7r3
- v7.3.35
- NEW: (#6643) Support OpenSSL 3.0.x
- Several fixes for AREX CE
- v7.3.36 should be done soon once existing PRs are merged in.
- v8r0
- Last one is v8.0.15
- CHANGE: (#6794) Speed up ReplicateAndRegister operation
- NEW: (#6785) Improve documentation on installing HTTPs services
- NEW: (#6708) SensitiveDataFilter in LogFilters
- v8r1
- Mostly fixes from the hackathon
DIRAC projects
- Federico I have “Archived” from GitHub a few obsoleted projects under https://github.com/DIRACGrid/
- One project was deleted (DIRAC4Android, last commit 10 years ago…)
- Others to be archived: VMDIRAC, BoincDIRAC, DIRACOS (?)
- RESTDirac got an update recently…?
DIRAC:
Issues by milestone:
Other issues:
PRs discussed:
WebApp:
- Few PRs open, connected to DIRAC PRs
Pilot:
DIRACOS2:
- from previous meeting Chris Is anyone using v7r2 client with py3?
- Still, for a few more days (CLIC), a bit longer (May, Belle2)
- Chris we might end up breaking v7r2 support
- will need to update to OpenSSL 3 early next year as OpenSSL 1.1.1 will reach EOL soon: https://github.com/DIRACGrid/DIRACOS2/pull/95
- 9th Feb Seems ready to go
- from previous meeting MySQL 8.0.27 version forced, as it drops support for old TLS
- and VOs using MySQL 5.7.9 or earlier? That would break with the next DIRACOS release
- The PR was merged
- We could potentially use python 3.10, or even 3.11 already
- Python 3.11 seems to be fine
- Propose to update immeidately after OpenSSL 3
- Opened an annoucement about OpenSSL 3
- Seen a couple of sites which don’t work with OpenSSL 3’s more secure defaults
- Included a script to allow others to check if they have affected sites
- Please try it and report the results (and ticket sites if needed)
- Do we need
myproxy in DIRACOS?
Documentation:
OAuth2:
- from previous meeting
- TokenManager WebApp will be added (pretty much like ProxyManager page), plus testing
tornado/HTTPs
- from previous meeting Issue https://github.com/DIRACGrid/DIRAC/issues/6495 keeps track of what can and what will not be moved to https
- BundleDelivery: replaced by a completely different solution? Proposal accepted
- Federico we need to try and propose a coherent update from DIPS to HTTPS services, e.g. for Configuration Services URLs
- who already runs HTTPs services in production?
- apart from LHCb, only very few from EGI.
management
- from previous meeting 3 issues left, still valid
diraccfg
- Dropped support for py2 in https://github.com/DIRACGrid/diraccfg/pull/7
- Few PRs followed with related updates
- Added full type hints so
mypy --strict passes - Chris B Plan to do a little more clean up with the packaging and then make a 1.0.0 release
- Would then be picked up in the following DIRACOS2 version
COMDIRAC
Daniela
DB12
Alexandre
Rucio
Tests
- Moving to HTTPs services as defaults in PR
- After this, even the “DIPS” tests will use a few HTTPs services, at least the Master CS
- Within this PR we will also:
- Rename CS sections of https services from
Systems/../Service/FileCatalog to Systems/../Service/TornadoFileCatalog for added clarity - Update the instructions on how to deploy
Release planning, tests and certification
Certification machines
- lbcertifdirac70 machine:
- Computing Element of each supported type should be configured (HTCondorCE, 2 ARCS (Alexandre (maybe use RALPP and Glasgow ? I can talk to them if there are issues --Daniela)), 1 Cloud (Daniela - this is already part of the tests), SSH). Consider OAuth tests as well - what is needed as far as certification servers configuration is concerned.
- Daniela: Enabled (and tested, at least with X509) WLCG VO for token testing. Needs update in puppet for /etc/grid-security/vomsdir and /etc/vomses, please.
- lbcertifdiracoauth machine:
Next hackathon(s)
- March 2nd on lbcertifdirac70
AOB
Next hackathon: March 2nd
Next BiLD: March 9th
LHCbDIRAC
- v11.0: deploy board in https://trello.com/b/Ep0PAkbv/deploy-110
- https://lhcb-auth.web.cern.ch/
- from previous meeting Andrei this is not properly configured yet. Not usable for Pilot submission yet (compute scopes not added).
- Alexandre An admin can allow a user (a client actually) to get access to the
compute.cancel/create/modify scopes. Chris allowed Andrei and I to get access to the compute scopes, thus we should be able to use our clients to submit pilots.
- Bookkeeping …
- Singularity CE everywhere?
- Need to start using multicore queues – one for each Site where we have storage. task to be added to Trello
- memory usage of a single core job of MCReco
There are minutes attached to this event.
Show them.
