Bi-weekly DIRAC Development meeting -- 31/01/2019

At CERN: Federico, Zoltan, Christophe, Vladimir, Marko, Sasha
On Vidyo: Hideki, Ueda, Daniela, Simon, Andrei, Marcelo, Mathew, Wojciech

Apologies:

Follow-up from previous meeting

• Created v6r21 branch (no release yet) from v6r20

DIRAC communities roundtable

• CLIC:
  • NTR
• LHCb:
  • NTR
• GridPP:
  • v6r20p26 in production
• Belle2:
  • Still running very old v6r17
    • going to upgrade soon (early next year) to v6r20
  • Encountered issues with dependency from boost, asked some sites to intervene manually
    • it will be solved with DIRACOS
    • using very old bundle does not help
• EGI:
  • working on "CheckIn" solution for SSO
    • OAuth
  • incorporating in DIRAC targeting v7r0
    • On-the-fly creation of VOMS certificates
  • WebPortal is updated, but needs to also work on server side now
    • create a token from CLI is needed, like creating a proxy
• IGI:
  • Just started with Virgo, km3net, Icarus
    • first test jobs fine
• PNNL
  • Supporting a few experiments
  • still using old DIRAC version

Current situation

DIRAC

• made v6r20p26 with the few bug fixes PRs that we have right now
  • maybe the last patch of v6r20 release series
• moved more problematic PRs to new release v6r21 that includes:
  • PR for FTS #3922 (requires DB changes and running a script)
  • PR for modifying the PYTHONPATH of pilots #3923
  • backport of dirac-install from integration branch, with a couple more fixes that will come
    • this will give us possibility to install optionally DIRACOS
  • 3 pre-releases up to now
  • plan to release very soon

WebApp:

• Just one PR to be included in the next production release (being it v6r20p27 or v6r21)

Pilot3:

• Federico added mostly tests, and portings
• Wojciech and Danyil have a minimal working prototype for the rest https tornado-based interface for pilot logging
  • can interface to MQ
  • for the moment kept in a separated repository
  • consumers in DIRAC (MySQL DB ready) or maybe using ES

DIRACOS:

• Fixing the upload of tarballs to EOS
  • Using http://diracos.web.cern.ch/diracos/
    • maybe one day will converge to using the same URL of DIRAC tarballs
  • gitlab-CI used -- in repo soon
  • Release creation being fully automated with it
• DIRACOS v1r0 should be ready, tag soon (now for real)

Documentation

• new script (v7r0) for creating the documentation of the "dirac-*" commands, using the "--help" switch is in integration branch now
  • We can put more docs in the docstrings, like done for dirac-install
• new howto section in PR (Chris)
• No updates to website

BOINC:

• NTR

DB12:

• NTR

tornado and other externals

• Chris tried M2Crypto from Simon, looks all good and promising

Release planning, tests and certification

Release planning:

• v6r20 series
  • v6r20p26 probably the last patch of v6r20
• v6r21 series
  • -pre3 ready
• v7r0 series
  • Should install on DIRAC certification with:
    • DIRAC v7r0-preX
    • DIRACOS  v1r0
    • WebApp 4.0
    • releases.cfg should be modified accordingly [done?]
  • bashrc will need to be re-done, so a fresh installation is needed
    • Instructions should go to https://github.com/DIRACGrid/DIRAC/wiki/DIRAC-v7r0
  • All Jenkins tests should be updated

Certification process: 

• The DIRAC certification machine should be re-done from scratch, including the DB and all
  • use CentOS7
• Started looking into Jenkins pipelines, looks good and powerful but not easy

Weekly development(s) focus

• NTR

DIRAC: current PRs and tasks being worked on, or topics from Google forum

• NTR

AOB