BiLD-Dev

Europe/Zurich
2-R-014 (CERN)

2-R-014

CERN

Federico Stagni (Conseil Europeen Recherche Nucl. (CERN)-Unknown-Unknown)
Description
Bi-Weekly "Loyal" DIRAC developers meeting. And, following, the LHCbDIRAC developers meeting.

Bi-weekly DIRAC Development meeting -- 31/01/2019

 

At CERN: Federico, Zoltan, Christophe, Vladimir, Marko, Sasha
On Vidyo: Hideki, Ueda, Daniela, Simon, Andrei, Marcelo, Mathew, Wojciech

Apologies:

 

Follow-up from previous meeting

  • Created v6r21 branch (no release yet) from v6r20

DIRAC communities roundtable

  • CLIC:
    • NTR
  • LHCb:
    • NTR
  • GridPP:
    • v6r20p26 in production
  • Belle2:
    • Still running very old v6r17
      • going to upgrade soon (early next year) to v6r20
    • Encountered issues with dependency from boost, asked some sites to intervene manually
      • it will be solved with DIRACOS
      • using very old bundle does not help
  • EGI:
    • working on "CheckIn" solution for SSO
      • OAuth
    • incorporating in DIRAC targeting v7r0
      • On-the-fly creation of VOMS certificates
    • WebPortal is updated, but needs to also work on server side now
      • create a token from CLI is needed, like creating a proxy
  • IGI:
    • Just started with Virgo, km3net, Icarus
      • first test jobs fine
  • PNNL
    • Supporting a few experiments
    • still using old DIRAC version

Current situation

DIRAC

  • made v6r20p26 with the few bug fixes PRs that we have right now
    • maybe the last patch of v6r20 release series
  • moved more problematic PRs to new release v6r21 that includes:
    • PR for FTS #3922 (requires DB changes and running a script)
    • PR for modifying the PYTHONPATH of pilots #3923
    • backport of dirac-install from integration branch, with a couple more fixes that will come
      • this will give us possibility to install optionally DIRACOS
    • 3 pre-releases up to now
    • plan to release very soon

WebApp:

  • Just one PR to be included in the next production release (being it v6r20p27 or v6r21)

Pilot3:

  • Federico added mostly tests, and portings
  • Wojciech and Danyil have a minimal working prototype for the rest https tornado-based interface for pilot logging
    • can interface to MQ
    • for the moment kept in a separated repository
    • consumers in DIRAC (MySQL DB ready) or maybe using ES

DIRACOS:

  • Fixing the upload of tarballs to EOS
    • Using http://diracos.web.cern.ch/diracos/
      • maybe one day will converge to using the same URL of DIRAC tarballs
    • gitlab-CI used -- in repo soon
    • Release creation being fully automated with it
  • DIRACOS v1r0 should be ready, tag soon (now for real)

Documentation

  • new script (v7r0) for creating the documentation of the "dirac-*" commands, using the "--help" switch is in integration branch now
    • We can put more docs in the docstrings, like done for dirac-install
  • new howto section in PR (Chris)
  • No updates to website

BOINC:

  • NTR

DB12:

  • NTR

tornado and other externals

  • Chris tried M2Crypto from Simon, looks all good and promising

Release planning, tests and certification

Release planning:

  • v6r20 series
    • v6r20p26 probably the last patch of v6r20
  • v6r21 series
    • -pre3 ready
  • v7r0 series
    • Should install on DIRAC certification with:
      • DIRAC v7r0-preX
      • DIRACOS  v1r0
      • WebApp 4.0
      • releases.cfg should be modified accordingly [done?]
    • bashrc will need to be re-done, so a fresh installation is needed
      • Instructions should go to https://github.com/DIRACGrid/DIRAC/wiki/DIRAC-v7r0
    • All Jenkins tests should be updated

Certification process

  • The DIRAC certification machine should be re-done from scratch, including the DB and all
    • use CentOS7
  • Started looking into Jenkins pipelines, looks good and powerful but not easy

Weekly development(s) focus

  • NTR

DIRAC: current PRs and tasks being worked on, or topics from Google forum

  • NTR

AOB

There are minutes attached to this event. Show them.
    • 10:00 10:10
      Items from Previous BiLD-Dev 10m
    • 10:10 10:20
      DIRAC Communities roundtable 10m
    • 10:20 10:35
      Current situation 15m

      General remarks

      • DIRAC 3m
      • WebApp 2m
      • Pilot 2m
      • DIRACOS 2m
      • Documentation 2m
      • DB12, BOINC 2m
      • tornado, M2Crypto, other externals 2m
    • 10:35 10:50
      Release planning, tests and certification 15m
    • 10:50 11:10
      Weekly development(s) focus 20m
    • 11:10 11:20
      DIRAC: current PRs and tasks being worked on 10m

      Ongoing PRs
      - v6r21 PR
      Ongoing tasks
      - ?
      Topics from the google forum
      - ?

    • 11:20 11:30
      AOB
      Convener: Federico Stagni (CERN)
    • 11:30 11:45
      LHCbDIRAC 15m