BiLD-Dev
Bi-Weekly "Loyal" DIRAC developers meeting. And, following, the LHCbDIRAC developers meeting.
Join Zoom Meeting
https://cern.zoom.us/j/91083694183?pwd=ZkdDY1I1YkJVc2o3UTdBY1BRZE15UT09
Meeting ID: 910 8369 4183
Passcode: 12345678
One tap mobile
+41432107108,,91083694183# Switzerland
+41315280988,,91083694183# Switzerland
Dial by your location
+41 43 210 71 08 Switzerland
+41 31 528 09 88 Switzerland
+41 43 210 70 42 Switzerland
+33 1 8699 5831 France
+33 1 7037 2246 France
+33 1 7037 9729 France
Meeting ID: 910 8369 4183
Find your local number: https://cern.zoom.us/u/artUfAdNB
Join by SIP
91083694183@188.184.89.188
91083694183@188.185.118.153
Join by H.323
188.184.89.188
188.185.118.153
Meeting ID: 910 8369 4183
Passcode: 12345678
BiLD (Bi-weekly DIRAC Development meeting) – 25/02/2021
At CERN: Still, nobody :/
On Zoom: Federico, Alexandre, André, Andrei, Andrii, Cedric, Christophe, Christopher, Daniela, Marko, Simon, Vladimir, Xiaomei
Apologies:
Follow-up from previous meeting
-
Ran hackathon last week on v7r2-pre34:
- trello board here
- VMDIRAC not yet fully installed nor tried
- https didn’t show any problem (ran with DFC on https)
- In general, very smooth, only minor issues. We are almost ready for creating the final v7r2 release.
-
- André no time for follow-up
- Andrei the implications should be clearly stated in the PR. This gives a bit too much responsibility to the developers.
- Federico please put this point in the issue above
-
HTCondorCE submission and usage
- the submission string that we applied in LHCb looks like being accepted by the sites, so it is the default now (PR) since v7r1p30 for the case when the local schedd is not used.
DIRAC communities roundtable
GridPP:
Daniela
- NTR
CLIC:
André
- NTR. Still running on v6r22, Will update ASAP.
LHCb:
Federico
- Using
httpsfor TPC transfer everywhere is possible (most of storages in LHCb with the notable exclusion of RAL).
EGI:
Andrei
- NTR
France Grilles:
Andrei
- NTR
BES3/Juno:
Xiaomei
- Juno: interested in using
httpsprotocol for TPC. - Federico you would need at least DIRACOS v1r17, meaning DIRAC v7r1p28+.
- Xiaomei we are still on v6r22
- Chistophe
xrootdis “sort of” decommissioning in terms of TPC. At the same time I also ran in parallel to thehttptests thexrootdtests, and DIRAC supports it.
DIRAC releases
-
v7r1p30:
- Using DIRACOS v1r18
- from #4972
- CHANGE: add a condition in TimeLeft module to know whether the batch system relies on wallclock and/or cpu time
- ADD: HTCondorResourceUsage module to get the time left in CERN-specific HTCondor resources.
- Together with (#4956) JobAgent - resorting to calculate time left with info in possession
- (#4964) dirac-proxy-init always uploads the proxy unless
--no-uploadis specified.
-
v7r2-pre35:
- fixes, mostly for python3
FedericoI suggest we stop accepting patches to v6r22, and by default now to push to use v7r1 better than v7r0.
- problem is that still several installations are still on v6r22
DIRAC projects
DIRAC:
PRs
- PR on DEncode vs JEncode
- will be closed, and basically we will leave with
DEncodealso in python3, until we get bitten.
- will be closed, and basically we will leave with
dirac-installscript is being moved to the management repo. See https://github.com/DIRACGrid/DIRAC/pull/4994 and https://github.com/DIRACGrid/management/pull/32- proper advertisement needs to be done.
Issues
- Update dirac-install for DIRACOS2
- for python3 we can get rid of
dirac-install, anddirac-configurecan be made smarter. See comment in issue above.
- for python3 we can get rid of
- TLS protocol and DIRAC was still a probem for Belle, fixed now in v7r0
By milestone:
- v7r0:
- no issues left
- v7r1:
- almost only documentation is still needed
- v7r2:
- Selectively install console_scripts is being addressed in Setup.cfg: split server scripts from the rest
- We should update the list of commands, action on Andre and Federico, because e.g. admin commands should not need to be in the basic UI
- create
user,server,admin,pilottags
- Most of the other tasks are being addressed, and those that won’t be addressed now will be addressed in the coming patches.
- Selectively install console_scripts is being addressed in Setup.cfg: split server scripts from the rest
- v7r3:
- Make RSS VO-aware was moved from v7r2 to v7r3
- Janusz tests being run
- it affects the SiteDirector and the Matcher.
- Make RSS VO-aware was moved from v7r2 to v7r3
WebApp:
PRs
- A few PRs merged. The version for v7r1 could be bumped.
Issues
- A few issues closed
Pilot3:
- NTR
DIRACOS:
- https://github.com/DIRACGrid/DIRACOS/issues/166
- can be closed
- v1r18 created.
DIRACOS2:
fts-reston py3 should be really now in a few weeks.- but for the moment is removed from the python3 client, it will be put back
- Some nice release notes added (with diffs)
- Non-pre-release of DIRACOS2 will be done together with v7r2.
VMDIRAC:
- Dependencies added to DIRACOS+2
- pylint for python3 ran, several fixes added. Running pylint on both python versions in the CI.
Documentation:
- Couple PRs for docs, nothing else to reports.
OAuth2:
- WIP
- News from issue WLCG and Token transition – reminders and requirements where there are a few links of meetings and docs that WLCG is doing?
- Andrii : I have left a comment.
- Andrei: there’s a general move towards using token. Some communities specifically addressed, like LHCb. CMS developed a tool for moving from VOMS to Indigo IAM.
- but understanding will still be needed.
- they have quite an aggressive timescale basically moving in 2021.
tornado/HTTPs
- NTR
management
diraccfg
- NTR
other externals, including Rucio
- Rucio File Catalog implementation from Belle
- Went through a couple reviews. Still a few things to be addressed. Needs a review from GridPP
- Janusz I haven’t looked at it yet. We need to test it with RAL Multi-VO Rucio server instance.
Release planning, tests and certification
-
Certification machine
- VMDIRAC
- still WIP
- Reminder: task for multi-vo
- VMDIRAC
-
Next hackathon
- next week (4th March)
- Not sure if it will be for v7r2 or for v7r3
- next week (4th March)
Weekly development(s) focus
NTR
AOB and topics from Google forum
Next hackathon on March 4th: https://indico.cern.ch/event/1012617/
Next BiLD on March 11th: https://indico.cern.ch/event/1012616/
DUW10 online: https://indico.cern.ch/event/852597/overview 10-11th May 2021. Looks like “approved”, will be announced officially at the beginning of the next week.
LHCbDIRAC
- v10r2 planning in https://gitlab.cern.ch/lhcb-dirac/LHCbDIRAC/-/wikis/Releases-planning#based-on-dirac-v7r2
- this release does not need more developments on LHCbDIRAC side. So considering our schedule, we think that in ~2 months we would be ready to install it in production.
- we want anyway, before restarting to do hackathons to first update the certification setup with what we put in https://codimd.web.cern.ch/2es8HPUcSmS42jrTxhX3JQ?both
- which actually should be expanded by Chris and Chris
- MySQL 8: ticket open for upgrade. A couple tests were done by the DBoD service, a few minor issues discovered, one needs a relase of v7r2 (patch already done)
- ES7: slowly progressing