BiLD (Bi-weekly DIRAC Development meeting) – 25/02/2021

At CERN: Still, nobody :/
On Zoom: Federico, Alexandre, André, Andrei, Andrii, Cedric, Christophe, Christopher, Daniela, Marko, Simon, Vladimir, Xiaomei
Apologies:

Follow-up from previous meeting

Ran hackathon last week on v7r2-pre34:
- trello board here
- VMDIRAC not yet fully installed nor tried
- https didn’t show any problem (ran with DFC on https)
- In general, very smooth, only minor issues. We are almost ready for creating the final v7r2 release.
PRs merging strategy?
- André no time for follow-up
- Andrei the implications should be clearly stated in the PR. This gives a bit too much responsibility to the developers.
  - Federico please put this point in the issue above
HTCondorCE submission and usage
- the submission string that we applied in LHCb looks like being accepted by the sites, so it is the default now (PR) since v7r1p30 for the case when the local schedd is not used.

DIRAC communities roundtable

GridPP:

Daniela

CLIC:

André

NTR. Still running on v6r22, Will update ASAP.

LHCb:

Federico

Using https for TPC transfer everywhere is possible (most of storages in LHCb with the notable exclusion of RAL).

EGI:

Andrei

France Grilles:

Andrei

BES3/Juno:

Xiaomei

Juno: interested in using https protocol for TPC.
Federico you would need at least DIRACOS v1r17, meaning DIRAC v7r1p28+.
- Xiaomei we are still on v6r22
Chistophe xrootd is “sort of” decommissioning in terms of TPC. At the same time I also ran in parallel to the http tests the xrootd tests, and DIRAC supports it.

DIRAC releases

v7r1p30:
- Using DIRACOS v1r18
- from #4972
  - CHANGE: add a condition in TimeLeft module to know whether the batch system relies on wallclock and/or cpu time
  - ADD: HTCondorResourceUsage module to get the time left in CERN-specific HTCondor resources.
- Together with (#4956) JobAgent - resorting to calculate time left with info in possession
- (#4964) dirac-proxy-init always uploads the proxy unless --no-upload is specified.
v7r2-pre35:
- fixes, mostly for python3

FedericoI suggest we stop accepting patches to v6r22, and by default now to push to use v7r1 better than v7r0.

problem is that still several installations are still on v6r22

DIRAC projects

DIRAC:

PRs

PR on DEncode vs JEncode
- will be closed, and basically we will leave with DEncode also in python3, until we get bitten.
dirac-install script is being moved to the management repo. See https://github.com/DIRACGrid/DIRAC/pull/4994 and https://github.com/DIRACGrid/management/pull/32
- proper advertisement needs to be done.

Issues

Update dirac-install for DIRACOS2
- for python3 we can get rid of dirac-install, and dirac-configure can be made smarter. See comment in issue above.
TLS protocol and DIRAC was still a probem for Belle, fixed now in v7r0

By milestone:

v7r0:
- no issues left
v7r1:
- almost only documentation is still needed
v7r2:
- Selectively install console_scripts is being addressed in Setup.cfg: split server scripts from the rest
  - We should update the list of commands, action on Andre and Federico, because e.g. admin commands should not need to be in the basic UI
  - create user, server, admin, pilot tags
- Most of the other tasks are being addressed, and those that won’t be addressed now will be addressed in the coming patches.
v7r3:
- Make RSS VO-aware was moved from v7r2 to v7r3
  - Janusz tests being run
  - it affects the SiteDirector and the Matcher.

WebApp:

PRs

A few PRs merged. The version for v7r1 could be bumped.

Issues

A few issues closed

Pilot3:

DIRACOS:

https://github.com/DIRACGrid/DIRACOS/issues/166
- can be closed
v1r18 created.

DIRACOS2:

fts-rest on py3 should be really now in a few weeks.
- but for the moment is removed from the python3 client, it will be put back
Some nice release notes added (with diffs)
Non-pre-release of DIRACOS2 will be done together with v7r2.

VMDIRAC:

Dependencies added to DIRACOS+2
pylint for python3 ran, several fixes added. Running pylint on both python versions in the CI.

Documentation:

Couple PRs for docs, nothing else to reports.

OAuth2:

WIP
News from issue WLCG and Token transition – reminders and requirements where there are a few links of meetings and docs that WLCG is doing?
- Andrii : I have left a comment.
- Andrei: there’s a general move towards using token. Some communities specifically addressed, like LHCb. CMS developed a tool for moving from VOMS to Indigo IAM.
  - but understanding will still be needed.
- they have quite an aggressive timescale basically moving in 2021.

tornado/HTTPs

management

diraccfg

other externals, including Rucio

Rucio File Catalog implementation from Belle
- Went through a couple reviews. Still a few things to be addressed. Needs a review from GridPP
- Janusz I haven’t looked at it yet. We need to test it with RAL Multi-VO Rucio server instance.

Release planning, tests and certification

Certification machine
- VMDIRAC
  - still WIP
- Reminder: task for multi-vo
Next hackathon
- next week (4th March)
  - Not sure if it will be for v7r2 or for v7r3

Weekly development(s) focus

NTR

AOB and topics from Google forum

Next hackathon on March 4th: https://indico.cern.ch/event/1012617/
Next BiLD on March 11th: https://indico.cern.ch/event/1012616/

DUW10 online: https://indico.cern.ch/event/852597/overview 10-11th May 2021. Looks like “approved”, will be announced officially at the beginning of the next week.

LHCbDIRAC

v10r2 planning in https://gitlab.cern.ch/lhcb-dirac/LHCbDIRAC/-/wikis/Releases-planning#based-on-dirac-v7r2
- this release does not need more developments on LHCbDIRAC side. So considering our schedule, we think that in ~2 months we would be ready to install it in production.
- we want anyway, before restarting to do hackathons to first update the certification setup with what we put in https://codimd.web.cern.ch/2es8HPUcSmS42jrTxhX3JQ?both
  - which actually should be expanded by Chris and Chris
- MySQL 8: ticket open for upgrade. A couple tests were done by the DBoD service, a few minor issues discovered, one needs a relase of v7r2 (patch already done)
- ES7: slowly progressing