Bi-weekly DIRAC Development meeting – 16/04/2020

At CERN: Nobody, of course!
On Vidyo: Federico, Andrei, Igor, André, Zoltan, Simon, Marko, Christophe, Alexandre, Christopher, Andrii, Daniela, Xiaomei, Vladimir, Janusz

Apologies:

Follow-up from previous meeting

PR #4469 merged (ProxyDB): tests OK again
MonitoringDB getting corrupted (DIRAC certification + LHCb certification)
- understood how to fix it, not why it get corrupted
- Framework/Monitoring still shaky in LHCb Certification setup and DIRAC certification
- should consider a flag to disable it completely, but it would be better understing why
Getting a bit closer to having a date for DUW 2021

DIRAC communities roundtable

GridPP:

v6r22 in production
v7r1 on pre-prod servers

CLIC:

Failure of 3rd party copy from castor to eos (using GFAL2_XROOT) : seems like a fix needed on GFAL2 side, but more investigation is needed

LHCb:

Plan to put v7r0 in prod next week

France Grilles:

BioMed and COVID-19 activities stalling ATM

EGI:

finally made OSG researches for WeNMR. Working pretty well. Configured 2 logical queues (1 for COVID-19 specific) with HTCondor specific parameters.
v7r0 in production
- with S3 storage element enabled but not fully in production

Juno/Bes3:

tried EOS. 3rd party copy with xrootd not possible. 2 SEs can’t communicate directly if they don’t talk the same protocol.
- askef if there will ever be a matrix of protocols: no

Nica:

not much activity, so started updating to v7r0. Looks like installing easier. Pilot3 working fine. Also moving to MySQL 5.7.

Current situation

DIRAC

v6r22:
- v6r22p26 is buggy
v7r0:
- v7r0p19:
  - new: introduced S3 storage
- v7r0p20:
  - new: WLCGAccountingHTTPJson occupancy plugin (should hopefully become the default)
  - fix for cleaning the DataFiles table when cleaning a Transformation
    - this is only for transformations that are being cleaned, so all previously existing old files there would still need to be cleaned by hand (see https://github.com/DIRACGrid/DIRAC/issues/3710)
v7r1:
- created the v7r1 tag, but not really: the currently existing PRs (mostly for docs) need to be rebased and then they will be included.

WebApp:

Pilot3:

DIRACOS:

Doc moved to RST, published in readthedocs
v1r11 tested in LHCbDIRAC, not yet included in any DIRAC release

VM:

Documentation

VMDIRAC documentation linked to the main DIRAC doc
DIRACOS readtheadocs created (diracos.readthedocs.io) but not yet linked to main one (a few entry points possible)

OAuth2:

tornado, M2Crypto and other externals

management

diraccfg

Release planning, tests and certification

Release planning:

v6r22 series
- [Federico] will fix the failing test (with a PR)
v7r0 series
- NTR
v7r1 series
- Merged #4549 to reduce the number of tests, then everyone else should rebase
- Hackathon last week pretty OK
v7r2 series
- the exact content will be fully discussed next time, but already existing PRs can go in there (and the Auth2 PRs should be rebased)

Weekly development(s) focus

Python 3 recommendations (PR https://github.com/DIRACGrid/DIRAC/pull/4521)

everyone’s invited to submit their own recommendations and experience
we need a strategy:
- split in small, manageable tasks
- start with seeing that the linter is OK
  - doing it directory by directory (Federico will start with one example)

DIRAC: current PRs and tasks being worked on, or topics from Google forum

GridPP: we have to register files of users/VOs in DIRAC (https://github.com/DIRACGrid/DIRAC/issues/4548). Need to work out better how the users work.
- this is anyway a quite general case: what can at the moment probably solved with a case-specific script might be generalized later

AOB

Next week: probably no hackathon

Next BiLD in 2 weeks.

LHCbDIRAC

In order to deploy v10r0:

CVMFS client (should be solved today?)
- Chris B should have enough karma
issue with the monitoring
- Chris will have another look
Ganga needs to release (https://github.com/ganga-devs/ganga/pull/1640)
- as they are using deprecated API calls (deprecated since very long)