BiLD-Dev

Thursday 5 Mar 2020, 10:00 → 11:45 Europe/Zurich

2-R-014 (CERN)

Federico Stagni (Conseil Europeen Recherche Nucl. (CERN)-Unknown-Unknown)

Bi-Weekly "Loyal" DIRAC developers meeting. And, following, the LHCbDIRAC developers meeting.

Bi-weekly DIRAC Development meeting -- 05/03/2020

 

At CERN: Federico, Zoltan, Christophe, Alexandre, Christopher, Andre', Marko
On Vidyo: Simon, Janusz, Andrei, Andrii, Hideki

Apologies: Ueda

 

Follow-up from previous meeting

• payload environment isolation
  • See https://github.com/DIRACGrid/DIRAC/issues/4480 (will be a topic for a next BiLD meeting)

DIRAC communities roundtable

• GridPP:
  • NTR
• CLIC:
  • moving to MySQL 5.7
• LHCb:
  • Moved all DIRAC boxes (only 1 left) to CC7, v6r22 + LHCbDIRACOS (based on DIRAC v1r5), and using a new puppet module
• France Grilles:
  • NTR
• EGI:
  • NTR
• Belle2
  • Moving to v6r22

Current situation

DIRAC

• v6r22:
  • fix to avoid trying to compile the web portal when installing DIRAC from the web interface
• v7r0:
  • v7r0p15 is the last one
    • New StorageOccupancy accounting backported from LHCbDIRAC (web part is still missing)
    • fully removed Oracle mentions

VM:

• NTR

WebApp:

• v4r0 is the series for DIRAC versions v6r22, v7r0, v7r1
• v4r1 will be used for DIRAC v7r2
  • several changes for adapting to OAuth2 (see slides in presentation)
  • web.cfg will be made optional, by default use dirac.cfg for all its content

Pilot3:

• 2 PRs for master to evaluate (Andrei)
• should add checksum for pilot.tar when it's uploaded (Federico)

DIRACOS:

• NTR

Documentation

• Missing readthedocs for VMDIRAC (needed? merge in DIRAC?)

OAuth2:

• See pres and weekly focus

tornado and other externals

• M2Crypto and VOMS issues to be sorted out before v7r1 is out

management

• PR for using dirac-distribution container for creating DIRAC extensions releases, now fully working also for DIRAC extensions. Publushed on docker hub and GitHub docker registry, used in LHCb gitlab-CI for creating release tarballs

diraccfg

• published on PyPi and conda forge, used in LHCb gitlab-CI
• python 2 and 3
• can think of being THE only way to call DIRAC CFG

 

Release planning, tests and certification

Release planning:

• v6r22 series
  • NTR
• v7r0 series
  • only fixes in pipeline
• v7r1 series
  • last hackathon on Tuesday 3/3. 2 issues:
    • PilotWrapper could not find the pilot files, addressed in #4475 (for v7r0)
    • M2Crypto and VOMS extensions: not seen, not working even in v7r0, need to understand why
  • One more will be necessary next Thursday, hopefully the last
  • PRs currently marked for v7r1 can be merged (after review) but no more things than this.
• v7r2 series
  • After discussion and presentation (see below) PRs #4405 and #4421 will be rebased on integration, evaluated and eventually merged
  • pyGSI will be removed, only M2Crypto will stay (#4481)
  • the first steps for JSON will be completed (issues to re-assign the milestones)
  • We will need to decide how to run the DIRAC certification (with X509 proxies?)

Certification process: 

• Working PR for being able to run the integration tests (currently run regularly in Github actions) from DIRAC extensions
  • being tested in LCHb (in GitLab CI)
  • a proper doc+explanation will be needed, one tutorial will be done at the DIRAC workshop
• Github actions now used also by the WebApp

Weekly development(s) focus

• Presentation from Andrii about OAuth (see link in agenda). In general well received, originated several questions/comments. A few issues created:
  • https://github.com/DIRACGrid/DIRAC/issues?q=is%3Aissue+is%3Aopen+label%3AOAuth2

DIRAC: current PRs and tasks being worked on, or topics from Google forum

• v7r0:
  • NTR
• v7r1:
  • Marked PRs that can go in (after review)
• v7r2:
  • 2 PRs (OAuth) to be rebased

#4463 (RSS DB handlers loaded dynamically) can be transformed into a generic mechanism for loading DB classes (https://github.com/DIRACGrid/DIRAC/issues/3848)

 

AOB

Next week: DIRAC hackathon.

Next BiLD in 2 weeks.

CoronaVirus update and DUW: travels to Japan are for the moment under scrutiny by CERN. DUW confirmed for the moment, but expecting updates.

 

=================

LHCbDIRAC

LHCbDIRAC hackathon on Tuesday at 14:00

SpaceToken -> StorageOccupancy accounting: need plan

 

10:00 → 10:10 Items from Previous BiLD-Dev

10:10 → 10:20 DIRAC Communities roundtable

10:20 → 10:35 Current situation

General remarks

Release notes

Releases - cfg

10:35 → 10:50 Release planning, tests and certification

DIRAC certification

• Certification setup
• Cert task board teamplate
• Slack channel

10:50 → 11:10 Weekly development(s) focus

OAuth2

11:10 → 11:20 DIRAC: current PRs and tasks being worked on

Ongoing PRs
- v6r22 PRs
- v7r0 PRs
- v7r1 PRs
- v7r2 PRs
Ongoing tasks
- ?
Topics from the google forum
- ?

DIRAC PRs

google forum

11:20 → 11:30 AOB

Convener: Federico Stagni (CERN)

DUW 10

11:30 → 11:45 LHCbDIRAC