BiLD-Dev

Europe/Zurich
Federico Stagni (Conseil Europeen Recherche Nucl. (CERN)-Unknown-Unknown)
Description
Bi-Weekly "Loyal" DIRAC developers meeting. And, following, the LHCbDIRAC developers meeting.

Bi-weekly DIRAC Development meeting – 16/04/2020

At CERN: Nobody, of course!
On Vidyo: Federico, Andrei, Igor, André, Zoltan, Simon, Marko, Christophe, Alexandre, Christopher, Andrii, Daniela, Xiaomei, Vladimir, Janusz

Apologies:

Follow-up from previous meeting

  • PR #4469 merged (ProxyDB): tests OK again
  • MonitoringDB getting corrupted (DIRAC certification + LHCb certification)
    • understood how to fix it, not why it get corrupted
    • Framework/Monitoring still shaky in LHCb Certification setup and DIRAC certification
    • should consider a flag to disable it completely, but it would be better understing why
  • Getting a bit closer to having a date for DUW 2021

DIRAC communities roundtable

GridPP:

  • v6r22 in production
  • v7r1 on pre-prod servers

CLIC:

  • Failure of 3rd party copy from castor to eos (using GFAL2_XROOT) : seems like a fix needed on GFAL2 side, but more investigation is needed

LHCb:

  • Plan to put v7r0 in prod next week

France Grilles:

  • BioMed and COVID-19 activities stalling ATM

EGI:

  • finally made OSG researches for WeNMR. Working pretty well. Configured 2 logical queues (1 for COVID-19 specific) with HTCondor specific parameters.
  • v7r0 in production
    • with S3 storage element enabled but not fully in production

Juno/Bes3:

  • tried EOS. 3rd party copy with xrootd not possible. 2 SEs can’t communicate directly if they don’t talk the same protocol.
    • askef if there will ever be a matrix of protocols: no

Nica:

  • not much activity, so started updating to v7r0. Looks like installing easier. Pilot3 working fine. Also moving to MySQL 5.7.

Current situation

DIRAC

  • v6r22:
    • v6r22p26 is buggy
  • v7r0:
    • v7r0p19:
      • new: introduced S3 storage
    • v7r0p20:
      • new: WLCGAccountingHTTPJson occupancy plugin (should hopefully become the default)
      • fix for cleaning the DataFiles table when cleaning a Transformation
  • v7r1:
    • created the v7r1 tag, but not really: the currently existing PRs (mostly for docs) need to be rebased and then they will be included.

WebApp:

  • NTR

Pilot3:

  • NTR

DIRACOS:

  • Doc moved to RST, published in readthedocs
  • v1r11 tested in LHCbDIRAC, not yet included in any DIRAC release

VM:

  • NTR

Documentation

  • VMDIRAC documentation linked to the main DIRAC doc
  • DIRACOS readtheadocs created (diracos.readthedocs.io) but not yet linked to main one (a few entry points possible)

OAuth2:

  • NTR

tornado, M2Crypto and other externals

  • NTR

management

  • NTR

diraccfg

  • NTR

Release planning, tests and certification

Release planning:

  • v6r22 series
    • [Federico] will fix the failing test (with a PR)
  • v7r0 series
    • NTR
  • v7r1 series
    • Merged #4549 to reduce the number of tests, then everyone else should rebase
    • Hackathon last week pretty OK
  • v7r2 series
    • the exact content will be fully discussed next time, but already existing PRs can go in there (and the Auth2 PRs should be rebased)

Weekly development(s) focus

Python 3 recommendations (PR https://github.com/DIRACGrid/DIRAC/pull/4521)

  • everyone’s invited to submit their own recommendations and experience
  • we need a strategy:
    • split in small, manageable tasks
    • start with seeing that the linter is OK
      • doing it directory by directory (Federico will start with one example)

DIRAC: current PRs and tasks being worked on, or topics from Google forum

  • GridPP: we have to register files of users/VOs in DIRAC (https://github.com/DIRACGrid/DIRAC/issues/4548). Need to work out better how the users work.
    • this is anyway a quite general case: what can at the moment probably solved with a case-specific script might be generalized later

AOB

Next week: probably no hackathon

Next BiLD in 2 weeks.


LHCbDIRAC

In order to deploy v10r0:

  • CVMFS client (should be solved today?)
    • Chris B should have enough karma
  • issue with the monitoring
    • Chris will have another look
  • Ganga needs to release (https://github.com/ganga-devs/ganga/pull/1640)
    • as they are using deprecated API calls (deprecated since very long)
There are minutes attached to this event. Show them.