BiLD-Dev

Europe/Zurich
Federico Stagni (CERN)
Description

Bi-Weekly "Loyal" DIRAC developers meeting. And, following, the LHCbDIRAC developers meeting.

Join Zoom Meeting
https://cern.zoom.us/j/91083694183?pwd=ZkdDY1I1YkJVc2o3UTdBY1BRZE15UT09

Meeting ID: 910 8369 4183
Passcode: 12345678
One tap mobile
+41432107108,,91083694183# Switzerland
+41315280988,,91083694183# Switzerland

Dial by your location
        +41 43 210 71 08 Switzerland
        +41 31 528 09 88 Switzerland
        +41 43 210 70 42 Switzerland
        +33 1 8699 5831 France
        +33 1 7037 2246 France
        +33 1 7037 9729 France
Meeting ID: 910 8369 4183
Find your local number: https://cern.zoom.us/u/artUfAdNB

Join by SIP
91083694183@188.184.89.188
91083694183@188.185.118.153

Join by H.323
188.184.89.188
188.185.118.153
Meeting ID: 910 8369 4183
Passcode: 12345678

 

BiLD (Bi-weekly DIRAC Development meeting) – 09/02/2023

On Zoom: Federico, Andrei, Alexey, Alexandre, Christophe, Christopher, Daniela, Domenico, Ewoud, Simon, Janusz, Xiaomei
Apologies: André

Follow-up from previous meetings

  • Last BiLD 2 weeks ago
    • all updates below
  • Hackathon planned (but did not happen) last week
    • “as usual” lately, the PRs were not ready.

DIRAC communities roundtable

LHCb:

Federico+Alexandre+Christophe+Christopher+Alexey

  • Running v8.0 in production for 3 weeks
  • from previous meeting Have been asked where we stand concerning token deployment
    • need quick-ish what’s being developed

EGI

Andrei

  • Not yet updated to v8.0, mostly DB updates needed first.

ILC/CLIC/FCC/Calice:

  • NTR

Belle2

  • NTR
  • from previous meeting Production to v7r2, but need to move to v8.0 before May, for HTCondor support, potentially also for Rucio support

GridPP:

Daniela+Simon+Janusz

  • Daniela Tested v7.3.35 in preprod, no blockers; will put it in production if I get a quiet minute.

juno

Xiaomei

  • Trying to move v8.0 in May, need to test out the token-based submission

Topics from GitHub/Discussions or Google forum

only un-answered topics:

  • NTR

DIRAC releases

  • v7r3
    • v7.3.33 and v7.3.34 and v7.3.35
      • several fixes for ARC and AREX
    • v7.3.35
      • more fixes (mostly WMS)
      • Adds support for OpenSSL 3.0.x
    • Needs to create a new release though
  • v8r0
    • Last one is v8.0.13
      • N interesting TR
  • v8r1
    • Haven’t done one in a while

DIRAC projects

DIRAC:

Issues by milestone:

Other issues:

  • 2 new issues added for documentation fixes

PRs discussed:

  • [8.0]Pilot submission with tokens
    • Reviewed with good suggestions. Andrei Some simplifications can go in, we don’t need much abstractions as anyway we would stick to only IAM and CheckIn, and interacting to them is rather similar/simple.
    • Somewhat urgent. Reminder to stick with what has been discussed in previous topic-specific meetings, specifically:
      • “Groups” are internal to DIRAC, tokens will not have such a notion
      • no oidc-agent needed nor want to
  • [8.0] Pilot stamps available in the pilot execution environment
  • Federico Some PRs of mine have been created long time ago and they are ready. Since no review is coming, I will merge them without review tomorrow, unless someone does one.
  • Chris and Chris Reviewing PRs containing several style-related commits mixed with real changes is difficult

WebApp:

  • Few PRs open, connected to DIRAC PRs

Pilot:

DIRACOS2:

  • from previous meeting Chris Is anyone using v7r2 client with py3?
    • Still, for a few more days (CLIC), a bit longer (May, Belle2)
    • Chris we might end up breaking v7r2 support
    • will need to update to OpenSSL 3 early next year as OpenSSL 1.1.1 will reach EOL soon: https://github.com/DIRACGrid/DIRACOS2/pull/95
    • 9th Feb Seems ready to go
  • MySQL 8.0.27 version forced, as it drops support for old TLS
    • and VOs using MySQL 5.7.9 or earlier? That would break with the next DIRACOS release
  • We could potentially use python 3.10, or even 3.11 already

Documentation:

  • NTR

OAuth2:

tornado/HTTPs

management

  • from previous meeting 3 issues left, still valid
    • NTR

diraccfg

  • from previous meeting Christopher Do we want to make a release that drops support for py2?
    • yes

COMDIRAC

Daniela

DB12

  • NTR

Rucio

  • NTR

Tests

Release planning, tests and certification

Certification machines

  • lbcertifdirac70 machine:
    • Computing Element of each supported type should be configured (HTCondorCE, 2 ARCS (Alexandre (maybe use RALPP and Glasgow ? I can talk to them if there are issues --Daniela)), 1 Cloud (Daniela - this is already part of the tests), SSH). Consider OAuth tests as well - what is needed as far as certification servers configuration is concerned.
    • Daniela: Enabled (and tested, at least with X509) WLCG VO for token testing. Needs update in puppet for /etc/grid-security/vomsdir and /etc/vomses, please.
      • Christophe I’ll check and add it
      • Daniela Still missing as of 14/12/22 A working config can be found at /cvmfs/grid.cern.ch/etc/grid-security/vomsdir/wlcg/ and /cvmfs/grid.cern.ch/etc/grid-security/vomses/wlcg-voms.cloud.cnaf.infn.it
  • lbcertifdiracoauth machine:
    • NTR

Next hackathon(s)

  • February 16th on lbcertifdirac70

AOB

Next hackathon: February 16th (Daniela will be away)
Next BiLD: February 23rd


LHCbDIRAC

  • v11.0: deploy board in https://trello.com/b/Ep0PAkbv/deploy-110
    • new dashboards added in Kibana, but issues for few of them. Issue created in DIRAC, followed up by Ewoud
  • New DMS dashboards in https://monit-grafana.cern.ch/d/Q78h6E-nz/home?orgId=46
    • Federico are these double checked? “safe” enough to be advertised?
  • https://lhcb-auth.web.cern.ch/
    • from previous meeting Andrei this is not properly configured yet. Not usable for Pilot submission yet (compute scopes not added).
    • fixed? Alexandre Apparently no, seems like the compute scope does not exist (compute.cancel/create/modify/read). Update: they don’t appear by default but they exist.
    • hackathon proposed this February? https://indico.cern.ch/event/1236132/ (it is today lol)
      • seemingly not for us, though
  • We should create a new production release as we are running with hotfixes. MRs seems to be all ready for merge.
  • MR for the Boookeeping: https://gitlab.cern.ch/lhcb-dirac/LHCbDIRAC/-/merge_requests/1354
    • add a proxy class for Bookkeeping (explanation in the MR itself)
    • should be reviewed commit by commit
    • will give the possibility to “safely” change the BookkeepingDB Oracle queries, that are otherwise rather painful to debug
    • Alexey seems like the correct idea.
 
There are minutes attached to this event. Show them.
    • 10:00 10:10
      Items from Previous BiLD-Dev 10m
    • 10:10 10:20
      DIRAC Communities roundtable 10m
    • 10:20 10:30
      DIRAC releases 10m
    • 10:30 10:55
      DIRAC projects 25m
      • DIRAC
      • WebApp
      • Pilot
      • DIRACOS
      • DIRACOS2
      • VMDIRAC
      • Documentation
      • OAuth2
      • tornado/HTTPs
      • other externals (include Rucio)
    • 10:55 11:00
      Release planning, tests and certification 5m
    • 11:00 11:15
      Weekly development(s) focus 15m
    • 11:15 11:25
      AOB
      Convener: Federico Stagni (CERN)
    • 11:25 11:40
      LHCbDIRAC 15m