BiLD-Dev
Bi-Weekly "Loyal" DIRAC developers meeting. And, following, the LHCbDIRAC developers meeting.
Join Zoom Meeting
https://cern.zoom.us/j/91083694183?pwd=ZkdDY1I1YkJVc2o3UTdBY1BRZE15UT09
Meeting ID: 910 8369 4183
Passcode: 12345678
One tap mobile
+41432107108,,91083694183# Switzerland
+41315280988,,91083694183# Switzerland
Dial by your location
+41 43 210 71 08 Switzerland
+41 31 528 09 88 Switzerland
+41 43 210 70 42 Switzerland
+33 1 8699 5831 France
+33 1 7037 2246 France
+33 1 7037 9729 France
Meeting ID: 910 8369 4183
Find your local number: https://cern.zoom.us/u/artUfAdNB
Join by SIP
91083694183@188.184.89.188
91083694183@188.185.118.153
Join by H.323
188.184.89.188
188.185.118.153
Meeting ID: 910 8369 4183
Passcode: 12345678
BiLD (Bi-weekly DIRAC Development meeting) – 26/01/2023
On Zoom: Federico, André, Andrei, Alexandre, Christophe, Christopher, Daniela, Hideki, Simon, Janusz, Ueda, Michel
Apologies:
Follow-up from previous meetings
- Last BiLD several weeks ago (in 2022)
- all updates below
- No hackathon for a rather long-ish time (we should have one next week)
DIRAC communities roundtable
LHCb:
Federico+Alexandre+Christophe+Christopher
- DIRAC v8.0 deployed in production this Monday
- Few fixes that escaped several hackathons needed, so PRs created and merged quickly, then created new releases.
- Prob with content of UserProfile DB, fix applied directly to the content of the DB
- Security loophole discovered for WebApp, fixed with PR already merged and release made
- Changes to Nginx config actually problematic, as not considering the possibility of an alias. A better version will come.
- Have been asked where we stand concerning token deployment
- need quick-ish what’s being developed
EGI
Andrei
- Preparing for the 8.0 migration (2 servers already migrated). A test server is deployed to test new releases and to work on tokens stuff.
- Added 3 or 4 more federated clouds using the Cloud CE. Running OK.
- Token-based access also fine.
ILC/CLIC/FCC/Calice:
André
- Not yet migrated to 7.3. Machines running, but services not yet in use from these machines.
- Technical Student starting Feb 1
Belle2
Hideki
- Production to v7r2, but need to move to v8.0 before May, for HTCondor support, potentially also for Rucio support
- Federico+Chris do not jump from v7r2 to v8.0
- issues opened yesterday (https://github.com/DIRACGrid/DIRAC/issues/6724 and https://github.com/DIRACGrid/DIRAC/issues/6723) should be already addressed.
GridPP:
Daniela+Simon+Janusz
- XRSLExtraString: AREX uses seconds, ARC uses minutes: https://ggus.eu/index.php?mode=ticket_info&ticket_id=160024
- might be worth putting a warning in the documentation
- Andrei maybe this can be coded?
- Simon I don’t think it would be better, it’s more for the configuration
- Alexandre Indeed, it is written in the ARC documentation here: https://www.nordugrid.org/arc/arc6/users/xrsl.html?highlight=walltime#walltime
- Intermittent problems with configuration updates: fails with the usual “versions differ”, needs restart of Configuration Server to clear
- Andrei do you have an auto-merge flag? If you do, then the update will happen in the backgroud
- Daniela No, but I don’t think this is the case. Also seen on our pre-prod server
- Date for next v7r3 release ? We intend to deploy promptly, mainly for ARC fixes.
Topics from GitHub/Discussions or Google forum
only un-answered topics:
- Few updates but nothing to be discussed
DIRAC releases
- v7r3
- The last v7r3 release was created in 2022, it’s time for a new patch as several fixes went in
- v8r0
- v8r1
- Have not done a new pre-release in a while
DIRAC projects
DIRAC:
Issues by milestone:
- v7r3:
- Still few open issues
- Add possibility to specify a DIRACOS2 version in SystemAdministrator and Pilot.
- often requested
- v8.0:
- 22 open issues
- Requirements and tasks for token-based pilot submissions
- PR created as a draft: https://github.com/DIRACGrid/DIRAC/pull/6580
- v8.1:
- 20 open issues
- Keep or remove BundleDelivery machinery?
- PR created, ready
PRs discussed:
- [8.0]Pilot submission with tokens
- Rather urgent. Reminder to stick with what has been discussed in previous topic-specific meetings, specifically:
- “Groups” are internal to DIRAC, tokens will not have such a notion
- no
oidc-agentneeded nor want to
- Rather urgent. Reminder to stick with what has been discussed in previous topic-specific meetings, specifically:
- [8.1] WMS: Working with usernames instead of OwnerDNs
- Waiting for other PRs to be merged first, starting from [8.1] feat: RequestManagementSystem: remove DN in favour of owner
WebApp:
- One patch for v5.0 created quickly after PR https://github.com/DIRACGrid/WebAppDIRAC/pull/715
- Few other PRs with “minor” fixes also created/merged
Pilot:
- Merged
develtomaster10 days ago after a successful 1-week trial in LHCb production - PR for removing dirac-install and py2 DIRAC client installations (to devel)
DIRACOS2:
- from previous meeting htcondor 9.0.x updated to fix scitokens support
- fixed
- Chris Is anyone using v7r2 client with py3?
- Still, for a few more days (CLIC), a bit longer (May, Belle2)
- Chris we might end up breaking v7r2 support
- will need to update to OpenSSL 3 early next year as OpenSSL 1.1.1 will reach EOL soon: https://github.com/DIRACGrid/DIRACOS2/pull/95
- MySQL 8.0.27 version forced, as it drops support for old TLS
- and VOs using MySQL 5.7.9 or earlier? That would break with the next DIRACOS release
Documentation:
- NTR
OAuth2:
- from previous meeting
- WLCG timeline document: https://zenodo.org/record/7014668#.YyLag9JBwQ9
- Meeting note: https://demo.hedgedoc.org/cWl2Y5MtTwSn4V5Uic_K5w
- Extreme MoU: https://demo.hedgedoc.org/wbG3LHEwRqunvpaBWV3qiw?view
tornado/HTTPs
- Issue https://github.com/DIRACGrid/DIRAC/issues/6495 keeps track of what can and what will not be moved to https
management
- from previous meeting 3 issues left, still valid
- NTR
diraccfg
- from previous meeting Christopher Do we want to make a release that drops support for py2?
- yes
- fine by GridPP
COMDIRAC
[name=Daniela]
- We think it’s more or less done() and integrating it would help people like him: https://github.com/DIRACGrid/DIRAC/discussions/6650 () We didn’t abandon VMDIRAC after the merge either.
- The discussion reached a conclusion that, while the PR could be merged without too many modifications, real changes to be done will be tracked in a DIRAC issue.
DB12
- NTR
Rucio
- NTR
Tests
- Added a backward-compatibility test with https://github.com/DIRACGrid/DIRAC/pull/6654
- integration.yml file would need to be updated for every new branch
Release planning, tests and certification
Certification machines
- lbcertifdirac70 machine:
- Computing Element of each supported type should be configured (HTCondorCE, 2 ARCS (Alexandre (maybe use RALPP and Glasgow ? I can talk to them if there are issues --Daniela)), 1 Cloud (Daniela - this is already part of the tests), SSH). Consider OAuth tests as well - what is needed as far as certification servers configuration is concerned.
- Daniela: Enabled (and tested, at least with X509) WLCG VO for token testing. Needs update in puppet for /etc/grid-security/vomsdir and /etc/vomses, please.
- Christophe I’ll check and add it
- Daniela Still missing as of 14/12/22 A working config can be found at /cvmfs/grid.cern.ch/etc/grid-security/vomsdir/wlcg/ and /cvmfs/grid.cern.ch/etc/grid-security/vomses/wlcg-voms.cloud.cnaf.infn.it
- lbcertifdiracoauth machine:
- NTR
Next hackathon(s)
*
AOB
Next hackathon: February 2nd
Next BiLD: February 9th
LHCbDIRAC
- v11.0: deploy board in https://trello.com/b/Ep0PAkbv/deploy-110
- new dashboards added in Kibana, but issues for few of them
- Federico I did not properly check any of them yet
- new dashboards added in Kibana, but issues for few of them
- New DMS dashboards in https://monit-grafana.cern.ch/d/Q78h6E-nz/home?orgId=46
- Federico are these double checked? “safe” enough to be advertised?
- https://lhcb-auth.web.cern.ch/
- from previous meeting Andrei this is not properly configured yet. Not usable for Pilot submission yet (compute scopes not added).
- from previous meeting Move from Jira to GitLab for issues
- Anyone would be against to move to using GitLab issues instead?
- For the existing ones: review, close if you can, or keep it to be moved for later.
- Jira history: not sure if it will be possible to keep it.