Bi-weekly DIRAC Development meeting – 02/04/2020

At CERN: Nobody, of course!
On Vidyo: Federico, Andrei, André, Zoltan, Janusz, Simon, Marko, Christophe, Alexandre, Christopher, Andrii, Daniela, Xiaomei

Apologies:

Follow-up from previous meeting

After DIRAC Coffe chat (replacement for hackathon):

• https://github.com/DIRACGrid/DIRAC/wiki/MySQL-5.7-(MariaDB-10.2) (MySQL 5.7 updates collection)
• https://github.com/DIRACGrid/DIRAC/pull/4520 (*2CS agents documentation)
• All code of v7r1 auto-pep8-ed : you need to rebase your PRs!
• DIRAC and COVID-19: efforts in FG, GridPP, EGI, WLCG

After last BiLD:

• DUW postponed 1 year: new date not confirmed (~clash with CHEP, which changed date…)
• VMDIRAC documentation built and linked from main DIRAC doc
• FTS3 and proxy renewal: Christophe opened a PR for fixing it, but FTS maybe going in a bit different direction: marked as WIP until it’s sorted out

DIRAC communities roundtable

GridPP:

• Testing v7r1 including Janusz’multi-VO metadata fixes (running v6r22 in production)
• Will test DIRACOS v1r11 once out (connected to https://github.com/DIRACGrid/DIRAC/issues/4480)

CLIC:

• Failure of 3rd party copy from castor to eos (using GFAL2_XROOT) with some combinations of DIRAC version and OS under investigation. Works on SL6 with dirac v6r22 + lcg bundles.

LHCb:

• v6r22 + DIRACOS everywhere (only Centos7 machines) + MySql 5.7 for all DBs
• close to deploy v7r0 in production

France Grilles:

• COVID-19 jobs: sites willing to support BioMed jobs but only for COVID-19 -> using DIRAC tags (not very scalable/elegant solution - thinking at a better solution with global tags)

EGI:

• WeNMR VO: application related to COVID-19 studies - quite large sandboxes - cyfronet created a S3 storage, Chris created a new plugin for supporting that (PR for v7r0, needs a DIRACOS release for boto)
• OSG proposing resources
  • proposing to not access directly the site, HTCondorCE would dispatch than to the resources

Juno:

NTR

Current situation

DIRAC

• v6r22:
  • v6r22p25: only fixes for MySQL 5.7
• v7r0:
  • v7r0p18: minor fixes only
• v7r1:
  • See PRs

WebApp:

NTR

Pilot3:

NTR

DIRACOS:

• v1r10 version to be created today
  • for supporting S3, should be used in conjuction with DIRAC v7r0 and S3 support
• v1r11: New RPATH version: instead of LD_LIBRARY_PATH use RPATH (see PR https://github.com/DIRACGrid/DIRACOS/pull/127)
  • Created a private DIRACOS version with it: went through a LHCbDIRAC certification with LHCbDIRACOS. Looked all fine.
  • should be almost exactly equivalent

VM:

NTR

Documentation

• Sphinx understand MD, but at the same time we should try to avoid to use MD, and always favor RST instead
  • pandoc can be used to move MD to RST, e.g. see https://github.com/DIRACGrid/DIRACOS/pull/129
  • we put the DIRACOS doc also on readthedocs and linked to main DIRAC doc (see PR above)

tornado, M2Crypto and other externals

• PR #4469: passing the tests. As soon as review OK, should be merged

OAuth2:

NTR

management

NTR

diraccfg

NTR

Release planning, tests and certification

Release planning:

• v6r22 series
  • NTR
• v7r0 series
  • S3 PR
• v7r1 series
  • several PRs waiting for being fully reviewed + passing all tests
• v7r2 series
  • NTR

Certification process:

No hackathon last week but we should have one this Thursday. Hopefully with v7r1 close to final.

Weekly development(s) focus

NTR

DIRAC: current PRs and tasks being worked on, or topics from Google forum

• v7r0:
  • NTR
• v7r1:
  • As at last BiLD: needs at least a documentation PR, for describing what has been introduced in v7r1, including the new concepts (e.g. “what’s an IdP?”)
    • for management
    • for developers
  • Once PR #4469 will be merged, all remaining PRs will need to be rebased on top
    • tests are failing in the other v7r1 + integration PRs because they need these fixes
• v7r2:
  • NTR

AOB

Next BiLD in 2 weeks.

LHCbDIRAC

Things missing before releasing v10r0:

• Zoltan: understand the issue with the Monitoring DB getting corrupted
  • in any case, document how to disable it fully
• Christophe: CVMFS deploy: looks like it might take longer time to properly release versions on CVMFS (dev is fine, but can’t be fully replicated)
• Christopher: LHCbDIRACOS based on v1r11 DIRACOS
• Federico: fix the broken workflow tests
• Federico: review the wiki at https://gitlab.cern.ch/lhcb-dirac/LHCbDIRAC/-/wikis/Releases-planning#lhcbdiracv10r0