Bi-weekly DIRAC Development meeting -- 11/04/2019

At CERN: Zoltan, Christophe, Andre', Marko, Sasha
On Vidyo: Federico, Daniela, Simon, Hideki, Xiaomei, Raja

Apologies: Andrei

Follow-up from previous meeting

• NTR

DIRAC communities roundtable

• CLIC:
  • NTR, not moved yet to v6r21
• LHCb:
  • Using v6r21 in production for 3 weeks, all fine
  • Added 1 machine using DIRACOS and running few services
• GridPP:
  • NTR
• Belle2:
  • Upgraded to v6r20
    • Just running through the various upgrades
• Juno:
  • Going to move to v6r21 next month

Current situation

DIRAC

• v6r20p28 created with a fix for SiteDirector (reported originally in https://github.com/DIRACGrid/DIRAC/issues/4014)
  • the fix has not been ported to v6r21 or integration branches, as this wants a better/proper fix
• v6r21:
  • v6r21p3 is the last patch release
    • improvements for MultiProcessor jobs
    • Question on MQ config in https://github.com/DIRACGrid/DIRAC/pull/4007#issuecomment-482053154

VM:

• NTR

WebApp:

• NTR

Pilot3:

• PR on pilot logger merged in devel
• Working on secure logging from python 2.6
• Working on automatic tests within Jenkins

DIRACOS:

• New tag 1.1.0 should be done today
  • and added in the releases.cfg for the next patch
• Looking for python3-compatible libraries
  • the only one not compatible looks to be multi-mechanize, will be removed from here
  • should check mysql-python vs mysql-connector
• A howto for doing the upgrade will be prepared
  • A "tutorial-like" talk will be done at the DUW

Documentation

• Andre' + Chris working on refactoring the scripts for creating the docs so that they can be used by the extensions too.
  • First PR on this came, should be ~ready by the DUW
• Can and should try to make a proposal for Google season of docs 2019: https://groups.google.com/forum/#!topic/hep-software-foundation-google-summer-of-code/LQOU8lKi0qg

BOINC:

• NTR

DB12:

• NTR

tornado and other externals

• NTR

Release planning, tests and certification

Release planning:

• v6r21 series
  • NTR
• v7r0 series
  • Should install on DIRAC certification with:
    • DIRAC v7r0-preX
    • DIRACOS  v1r1
    • WebApp 4.0
    • releases.cfg should be modified accordingly [done?]
  • bashrc will need to be re-done, so a fresh installation is needed
    • Instructions should go to https://github.com/DIRACGrid/DIRAC/wiki/DIRAC-v7r0
  • All Jenkins tests should be updated
    • And possibly use the M2Crypto flag.
  • Chris also working with first integration tests run in GitLabCI

Certification process: 

• The DIRAC certification machine being re-done from scratch
  • CentOS7
• Marko asked about proper way of identifying the tag or branch a certain PR is building on
  • DIRAC.__init__.py may be used, but often it is not updated
  • setup.py file is there but not updated
  • will create an automated test to verify that this is correctly updated
    • in the longer term we should use gitlab-CI to help in creating the releases

Weekly development(s) focus

NTR

DIRAC: current PRs and tasks being worked on, or topics from Google forum

• Some changes discussed/agreed on calculating the space occupancy by Belle2 should be added to v6r20

AOB

DUW agenda ready

Next BiLD in 3 weeks from now.

---------------------------------------------------------------------------------------------------------------

LHCb

• Issues with BulkSubmission: bad merge? To be verified
• Alias for setting up LHCbDIRAC environment: discussed/agreed in https://its.cern.ch/jira/browse/LBCORE-1420
  • The lb-run hack/alias will stay for some time