Bi-weekly DIRAC Development meeting -- 28/03/2019

At CERN: Zoltan, Christophe, Andre', Sasha, Wojciech
On Vidyo: Federico, Andrei, Daniela, Simon, Hideki, Xiaomei, Raja

Apologies:

Follow-up from previous meeting

• NTR

DIRAC communities roundtable

• CLIC:
  • NTR, not moved yet to v6r21
• LHCb:
  • Using v6r21 in production, all OK
• GridPP:
  • NTR
• Belle2:
  • Still running very old v6r17
    • going to upgrade to v6r20 next week
• Juno:
  • FTS3+TS+RMS being tried out for DDM system, all looks fine.
    • Suggested to move to v6r21 as upgrade is easy and would avoid any FTS3 issues
  • Would need new VMDIRAC version, changes done for v7r0
• EGI:
  • Smooth operations
    • Working for DIRAC communities, including cloud sites
    • Pressures to use MP resource, mostly on VMs. Solution will be provided with the Local CE not being "InProcess", probably going to  implement what's already discussed in 
  • Ongoing development with the EGI check-in and authentication in the portal. Being tested now.
  • Working on using a "proxy provider" solution with 3 providers:
    • "DIRAC CA" for internal-only authentication
    • User subproxies
    • MyProxy (in there since long time)
      • Has to consider the changes in M2Crypto PR

Current situation

DIRAC

• v6r20p26 still the last one, won't commit to v6r20 anymore
• v6r21:
  • v6r21p2 is the last patch release

VM:

• libcloud being obsoleted, new connector for OpenStack, occi connect becoming obsolete too
  • need a new release

WebApp:

• NTR

Pilot3:

• Wojtek: PR for re-factorization of pilot logger. Hiding the way we do the logging is done.
• Working on secure logging from python 2.6
• Working on automatic tests within Jenkins
• Daniel adapting part of the code using tornado (for syncing DNs from CS).
  • almost ready

DIRACOS:

• v6r21 can install DIRACOS
• Being tried out, for server and client installation, and also for CVMFS deployed versions
• A howto for doing the upgrade will be prepared

Documentation

• Few updates to website
• Pointed to pycon talk on documentation: watch it and we should try to follow
  • push for doc before the DUW
• Andre' + Chris working on refactoring the scripts for creating the docs so that they can be used by the extensions too.

BOINC:

• NTR

DB12:

• NTR

tornado and other externals

• M2Crypto PR merged in integration

Release planning, tests and certification

Release planning:

• v6r21 series
  • NTR
• v7r0 series
  • All BIG PRs merged
  • Should install on DIRAC certification with:
    • DIRAC v7r0-preX
    • DIRACOS  v1r0
    • WebApp 4.0
    • releases.cfg should be modified accordingly [done?]
  • bashrc will need to be re-done, so a fresh installation is needed
    • Instructions should go to https://github.com/DIRACGrid/DIRAC/wiki/DIRAC-v7r0
  • All Jenkins tests should be updated
    • And possibly use the M2Crypto flag.
  • Chris also working with first integration tests run in GitLabCI

Certification process: 

• The DIRAC certification machine should be re-done from scratch
  • Federico created new DB, still to be configured
  • should use CentOS7
• Jenkins pipelines:
  • working fine for LHCbPilot

Weekly development(s) focus

Agreed on removing runsv from DIRACOS, instructions for first installation will need to be updated accordingly. The rpm will need to be taken from lhcb repo.

DIRAC: current PRs and tasks being worked on, or topics from Google forum

• https://github.com/DIRACGrid/DIRAC/issues/3990 discussed, few solutions proposed but no final decision.

AOB

CHEP2019 abstract submissions are open

---------------------------------------------------------------------------------------------------------------

LHCb

First working on bootstrapping DIRAC v7r0 certification, then focusing on v9r4.