Bi-weekly DIRAC Development meeting – 19/03/2020

At CERN: Nobody, of course!
On Vidyo: Federico, Andre, Chris Burr, Janusz, Marko, Andrei, Xiaomei, Hideki, Chris, Alexandre, Andrii, Daniela, Simon, Zoltan

Apologies: Wojtek

News

The DIRAC Users’ Workshop has been cancelled/postponed 1 year. The indico page has been updated, and there’s already a new date: start 24/05/2021, which is exactly 1 year after the planned one.

Follow-up from previous meeting

Nothing that needed a follow-up

DIRAC communities roundtable

GridPP:

Needed Janusz’ multi-VO fix for the DB.
Trying to move directly to v7r1 (suggested to go to v7r0 first)

• Proxy issues (needs proper documentation)
• Transformation System could not work properly (see https://groups.google.com/forum/#!topic/diracgrid-forum/bOv6QjPfqIg)

CLIC:

Moved to Mysql 5.7: https://github.com/DIRACGrid/DIRAC/pull/4503

LHCb:

• Moved all remaining DBs (DFC + TransformationDB) to MySQL 5.7: https://github.com/DIRACGrid/DIRAC/pull/4503/commits/96759aa12f5fdf9b5991770eb3ce7f256151fb97
• All (but 1) machines are now CentOS7 + v6r22 + DIRACOS

France Grilles:

• Lab in Marseille working on Covid-19, a special group created with high priority (within the BioMed community)
• Running v7r0, M2Crypto flag off, still Pilot2.

EGI:

EISCAT community using Ubuntu as server for file server. v7r0 with DIRACOS works fine. Highlighted that, while DIRAC services would work without issues, other DIRAC agents and services that require externals might not (side note: DIRAC integration tests in GitHub actions run in Ubuntu).

Belle2

Moving to v6r22, trouble building WebApp. One of the reasons is that they have Belle2WebApp code in the same repository of Belle2DIRAC: this is not the normally suggested way, and dirac-distribution doesn’t understand that. The suggestion is to use the dirac-distribution docker container that can be found also in docker hub (which works for DIRAC v6r22+).

Juno

Pressure test on TransformationSystem. Proxy lifetime with FTS issue: the proxy expires.
Chris: the proxy is renewed by FTS3 forcedly only when there’s less than 1 hour --> this is an FTS3 issue.

Current situation

DIRAC

• v6r22:
  • NTR
• v7r0:
  • Last one is v7r0p17
    • pilot tarball + pilot.json file subject to checksum checks
    • StorageOccupancy accounting now have a web part
• v7r1:
  • One hackathon done last week, one more next week

WebApp:

• Support for StorageOccupancy accounting added in latest v4r0 patch
• Added (in PR, checked by GitHub Actions) a prettier check (style checker for JS). A linter is also there (for JS).

Pilot3:

Wojtek working on PR for testing things with GitHib Actions

DIRACOS:

Tests are failing - looking at that [Marko P]

VM:

NTR

Documentation

• https://vmdirac.readthedocs.io/en/latest/
  • Needs a few tiny tweaks
    • https://github.com/DIRACGrid/VMDIRAC/pull/125
  • Webhook from vmdirac repo to-do

OAuth2:

The extension should be merged into DIRAC for v7r2 (all in one PR).

tornado, M2Crypto and other externals

Tests should pass in https://github.com/DIRACGrid/DIRAC/pull/4469/checks

management

NTR

diraccfg

NTR

Release planning, tests and certification

Release planning:

• v7r0 series
  • NTR
• v7r1 series
  • needs at least a documentation PR, for describing what has been introduced in v7r1, including the new concepts (e.g. “what’s an IdP?”)
    • for management
    • for developers
  • existing PRs can all be merged when the tests will be passing
  • Janusz’s PR for multiVO DFC can go in as soon as the tests are passing
• v7r2 series
  • not su much discussed today

Certification process:

As of today, we have too many integration tests done for each commit (the matrix got a few variables lately). Some will be dropped, others can be done only for a tag, or when pushing to a DIRACGrid/DIRAC branch [Chris B]

Weekly development(s) focus

Discussion on payload environment isolation: see https://github.com/DIRACGrid/DIRAC/issues/4480
Pretty neat summary, together with actions plan. Chris Burr will create a dev version of DIRACOS and then will be tested by Daniela and others in GridPP.

DIRAC: current PRs and tasks being worked on, or topics from Google forum

• v7r0:
  • NTR
• v7r1:
  • Just a few PRs waiting for the tests to pass
• v7r2:

PRs #4492 (style for DMS, Federico) and #4498 (style for all the code): the question is basically when to apply the second (style for all the code). After some considerations, we decided to close both these PRs, and soon, in a “calm” moment code-wise (e.g. when there are no other PRs open for v7r1) to apply this styling to rel-v7r1, create a pre-release with that only. Then all the other PRs will need to be rebased.

AOB

Next week we can have another DIRAC Hackathon for v7r1

Next BiLD in 2 weeks.

LHCbDIRAC

Hackathon of LHCbDIRAC v10r0-pre14 last monday

• Zoltan can’t find what’s the issue with Framework/Monitoring sqlite file
  • only checked that reading is still OK, suggested to try also to read to see if the file gets corrupted
• The last machine with SLC6 (master CS) will be moved today or next week