Bi-weekly DIRAC Development meeting -- 22/02/2019

At CERN: Federico, Zoltan, Christophe, Vladimir, Marko, Andre', Sasha, Marcelo
On Vidyo: Ueda, Simon, Xiaomei, Raja

Apologies:

Follow-up from previous meeting

• NTR

DIRAC communities roundtable

• CLIC:
  • Users experiencing minor issue with deprecated messages, fixed in https://github.com/DIRACGrid/DIRAC/pull/3966
• LHCb:
  • Working for introducing v6r21 in production ASAP
• GridPP:
  • NTR
• Belle2:
  • Still running very old v6r17
    • affected by the move of backend from AFS to EOS
      • in EOS backend (diracproject.web.cern.ch) we have only the last 2 patches of v6r17, they were running an older one and had to update
    • going to upgrade soon ? to v6r20 or v6r21?
• IGI:
  • Just started with km3net, Icarus
    • test jobs fine, improving
• BES:
  • Using GPU "federation", matching from DIRAC side (v6r20 series) looks fine

Current situation

DIRAC

• v6r20p26 still the last one
• v6r21:
  • actively tested by LHCb, still some issues
  • plan to release very soon

WebApp:

• NTR

Pilot3:

• Federico added mostly tests, and portings

DIRACOS:

• v1r0 out
  • gitlab-CI used for fully automating the release procedure
  • Using http://diracos.web.cern.ch/diracos/

Documentation

• new howto section in PR (Chris to move it to v6r21)
• No updates to website

BOINC:

• NTR

DB12:

• NTR

tornado and other externals

• Chris is using M2Crypto changes from Simon, and working on a decently large PR. 

Release planning, tests and certification

Release planning:

• v6r20 series
  • v6r20p26 still the last patch of v6r20
• v6r21 series
  • -pre7 ready, still buggy and some fixes in PRs
• v7r0 series
  • Should install on DIRAC certification with:
    • DIRAC v7r0-preX
    • DIRACOS  v1r0
    • WebApp 4.0
    • releases.cfg should be modified accordingly [done?]
  • bashrc will need to be re-done, so a fresh installation is needed
    • Instructions should go to https://github.com/DIRACGrid/DIRAC/wiki/DIRAC-v7r0
  • All Jenkins tests should be updated

Certification process: 

• The DIRAC certification machine should be re-done from scratch, including the DB and all
  • Federico needs to create new DB
  • use CentOS7
• Jenkins pipelines: powerful but not easy, implemented for LHCbPilot 3 and could be ported at least for Pilot3
  • we are anyway thinking to move one day fully to GitLab-CI, and we may have a student for it

Weekly development(s) focus

Moving webserver hosting tarballs from AFS to EOS backend (from lhcbproject.web.cern.ch to diracproject.web.cern.ch). Already using everything from EOS with the exclusion of the globalDefaults.cfg file, which is still linked from dirac-install in AFS. We can move (PR from Zoltan) but the old one will need to stay until we support v6r20 series.

DIRAC: current PRs and tasks being worked on, or topics from Google forum

• https://github.com/DIRACGrid/DIRAC/pull/3158 restored, to use concurrent.futures library in DIRAC services instead of DIRAC threadpool -- questions on how urgent (not urgent) and if a flag for enabling it is needed (maybe)
• https://github.com/DIRACGrid/DIRAC/pull/3962 (v7r0) introduces a pylint for pilot 3 check. Many files with minor modifications. Do not merge immediately or conflicts will arise with M2Crypto PR from Chris that will come.
• https://github.com/DIRACGrid/DIRAC/pull/3968 removes the old logging, but question to Andre' about seriealization...

AOB

DIRAC poster ready, can be downloaded and printed and attached in your institute! https://drive.google.com/file/d/1L0_moK5WZkM7phS7jjxwHY6gocmMIIj1/view