BiLD (Bi-weekly DIRAC Development meeting) – 25/06/2020

At CERN: Nobody, of course!
On Vidyo: Federico, Andrii, André, Christopher, Christophe, Daniela, Hideki, Janusz, Alexandre, Simon, Marko, Andrei, Vladimir

Apologies:

Follow-up from previous meeting

• Pilot3 and user jobs needing to run dirac-commands:
  • new PR from Simon merged (linking a dirac.cfg, if not there): tested on the GridPP preprod server and it works for GridPP.
• Chris: in LHCb pip does not do a proper dependencies resolution (the six version was not up-to-date). Sorted out by specifying a specific version --> no follow up in DIRAC docs
• PyTests failing for PoolCE: suggested to remove the coverage computation (Andrei)
  • PR created, one bug discovered in the Pilot, to be checked if it affects also pilot 3

DIRAC communities roundtable

GridPP:

• Made some progress on HTCondorCEs (after some suggestions from André)
  • problem on both sides (DIRAC and HTCondorCEs)
    • full leghth of the proxy to get: under Resources -> Computing -> HTCondorCE had to add ExtraSubmitString = delegate_job_GSI_credentials_lifetime = 0
      The request to document this properly is noted by GridPP.
• Waiting for CVMFS uploads from Andrei T. (This is to be able to create a pull request for GridPP’s v7 fixes to cloudinit, which in v6 relied on some version of dirac-install that happened to be on cvmfs. This was one of those “it’s a hack, we’ll fix it properly later issues, and we have now reached ‘later’”.)

CLIC:

• Finished migrating servers (CC7, DiracOS, v6r22)

LHCb:

• Testing v7r1 in certification. Pilot proxy not found. Need some help in sorting things out, so Andrei and Andrii will be invited to the next LHCbDIRAC hackathon
• M2Crypto: on 2 machines. 2 issues are actually inside M2Crypto (see below)

France Grilles:

• NTR. Looking into updating to Pilot3.

EGI:

• NTR

Belle2:

• Migration to v6r22 still pending.

Nica:

• NTR

Juno:

• NTR

Current situation

DIRAC

• v6r22:
  • NTR
• v7r0:
  • created v7r0p27
    • just a small fix (using transactions when creating DB tables)
• v7r1:
  • v7r1p5 created
    • just a small fix (Install of ES DBs: only search in *DB.py files)
• v7r2:
  • NTR

WebApp:

• Checked all tasks, re-assigned some (Andrii + Federico)

Pilot3:

• Linking etc/dirac.cfg to pilot.cfg : done

DIRACOS:

• xroot5?
  • new gfal release to be checked if that might solve the current issues

VM:

• NTR

Documentation:

• Just one issue asking for documentation for v7r1 https://github.com/DIRACGrid/DIRAC/issues/4654

OAuth2:

• Big PR in WIP: https://github.com/DIRACGrid/DIRAC/pull/4650 from Andrii
  • This PR includes the OAuthDIRAC extension.
  • The WebAppDIRAC for OAuthDIRAC extension will go in WebApp
  • Too early to discuss it today, will be a subject in a coming future

tornado and other externals

• M2Crypto: main maintainer MIA, the other 2 maintainers started reacting, after Chris contacted them. Chris opened 2 PRs in https://gitlab.com/groups/m2crypto/-/merge_requests and hotfixed in the LHCb setup, and there things work.
  • these fixes are today necessary, they won’t be necessary anymore once we’ll move to https
• tornado: Chris started working a bit more seriously on it. Created an equivalent service to the DFC. Will create a test for it.
  • Some JSON issues are still unresolved and blocking
  • the old PR from Luis will be replaced at some point by a new one (Chris rebased) but right now this one stays open for reference and comments

management

• Andrei: I have a couple scripts putting various DIRAC tarballs in CVMFS. Not yet added on the repository. Stand-alone administrator tools.
  • Marko: we deploy to CVMFS via GitLab, so this should be possible?
  • Andrei: I need to use a proxy
  • Andre’: you can store the proxy and use secrets

diraccfg

• NTR

Release planning, tests and certification

• Not much done wrt to what was discussed 3 BiLD meetings ago (which is still valid).

Weekly development(s) focus

NTR

DIRAC: current PRs and tasks being worked on, or topics from Google forum

PRs:

• v7r0:
  • GridPP https://github.com/DIRACGrid/DIRAC/pull/4651
• v7r1:
  • BDII2CSAgent and non-SRM SEs: not much to be done if Glue1/2 does not publish correctly
    • nothing done in the last 2 weeks on this topic
• v7r2:
  • Nothing that was not already discussed before

On issues:

• 2 more issues filed in since last BiLD

Topics from Google forum:

• NTR

AOB

Next BiLD in 2 weeks.

LHCbDIRAC

• Marko Petric will start within LHCb officially on July 1st, his program of work has been discussed at the end of the meeting

• BKK

  • Cert instance (INT12r) moved to 19c
    • need to update the instant client on the machine (in puppet) [Chris]
  • The prod server hardware will be upgraded Saturday
  • The prod server will be upgraded to 19c on July 1st
    • Federico will run tests (including performance tests – they look broken atm)
    • prob will ask to postpone 2 weeks

• Chris B: we can automatically install LHCbDIRAC in production

  • updates in lbtaskrun

• Vladimir: SiteDirector blocked from time to time. Might be linked to CEs being moved, we should not worry about.

• Alexandre: using of SDumont is blocked (Vladimir’s account blocked?)

  • mail sent out for clarifications