BiLD-Dev

→ Europe/Zurich
2-R-014 (CERN)

2-R-014

CERN

Federico Stagni (Conseil Europeen Recherche Nucl. (CERN)-Unknown-Unknown)
Description
Bi-Weekly "Loyal" DIRAC developers meeting. And, following, the LHCbDIRAC developers meeting.

Bi-weekly DIRAC Development meeting -- 18/07/2019

 

At CERN: Federico, Zoltan, Christophe, Andre', Marko, Christopher, Sasha, Wojciech
On Vidyo: Andrei, Andrii, Luisa, Xiaomei, Daniela, Simon, Hideki

Apologies: 

 

Follow-up from previous meeting

  • GS DOCS: google evaluating students. We got a few applications and selected 2. News at the end of the month
  • Closed several WebApp issues that are now in v4r0

DIRAC communities roundtable

  • CLIC:
    • moved to v6r21
  • LHCb:
    • NTR
  • CTA:
    • moved to v6r20 recently
    • new DB for provenance (postgresSQL), using sqlAlchemy for connecting to it. Had to install a python module (could be added in DIRACOS extension when available)
      • not registered as a catalog
  • GridPP:
    • Trying to connect DIRAC to Openstack cloud (questions on the forum)
    • Running v6r21p7
  • EGI
    • Running smoothly, and starting to use OAuth authentication, with some patches directly in production.
    • Incorporating new cloud resources, need to incorporate properly Pilot3 in the cloud
  • Belle2:
    • Moved to v6r21
  • Juno:
    • deploying production system (TS) and data movement (with TS)
    • queries done by the InputDataAgent to the DFC may become very slow (e.g. 800K files returned, way too much)
      • not many technical solutions, need to find an operational way around it
    • next week planning to upgrade to v6r21

Current situation

DIRAC

  • v6r21:
    • v6r21p12 is the last patch release
      • Minor fixes only in p11 and p12
  • v6r22:
    • v6r22 and v6r22p1 created
    • See the instructions in https://github.com/DIRACGrid/DIRAC/wiki/DIRAC-v6r22 for upgrading
      • not many new things
    • certified also with DIRACOS

VM:

  • NTR. Need to look into using Pilot3

WebApp:

  • v4r0p6 is the latest release
  • Created branch rel-v4r0
    • for v6r22 and v7r0
  • integration will go to v4r1 for v7r1 (next)
  • JSLint can be added for Travis tests [Zoltan]
    • needs node.js

Pilot3:

  • Merged devel into master, working fine in production for LHCb
  • One PR from Andrew for using MultiLauchAgent has been merged (also in master)
  • Wojciech is setting up a test for PilotLogger (with some fixes going into devel branch)
    • VM with tornado that sends messages to CERN ActiveMQ instance
    • Right now works only with python 2.7, plan to use system calls for python 2.6

DIRACOS:

  • Almost done with the script for extending DIRACOS
    • started with extending the python packages (this is the need right now)
  • v1r3 can be created

Documentation

  • NTR

OAuth2:

  • Will be used from DIRAC v7r1

DB12:

  • NTR

tornado and other externals

  • NTR

Release planning, tests and certification

Release planning:

  • v6r21 series
    • NTR
  • v6r22 series
    • NTR, only fixes
  • v7r0 series
    • a rel-v7r0 branch is going to be created including PRs:
      • #4160 (autopep8s...)
      • #4163 (fixes from last hackathon + Pilot3 is default)
      • #4170 (Production system docs and tests)
    • which will be for v7r0-pre19
  • Integration and v7r1:
    • PR #4157 (ProxyProviders and OAuth2) will be merged in integration and v7r1-pre1 created with it
    • All other PRs can for the moment wait (maybe #4103 and #4108 if updated, and #4166)

Certification process: 

  • Next week hackathon of v7r0-pre19

Weekly development(s) focus

  • DIRACOS extension: how to specify it in releases.cfg?
    • decided to use "DIRACOS = LHCb:vXrY" if it comes from an extension

DIRAC: current PRs and tasks being worked on, or topics from Google forum

  • Question from Janusz on RSS on the developer forum is answered

AOB

Decided to close the Google developer forum, and simply open github issues instead.

 

 

Next BiLD in 2 weeks from now.

 

---------------------------------------------------------------------------------------------------------------

LHCb

  • v9r4
    • second hackathon tomorrow
    • New bundle needed: include six plus new version of LbEnv
    • Needs DIRACOS v1r3 with fixed version of ElasticSearch
There are minutes attached to this event. Show them.
    • 10:00 → 10:10
      Items from Previous BiLD-Dev 10m
    • 10:10 → 10:20
      DIRAC Communities roundtable 10m
    • 10:20 → 10:35
      Current situation 15m

      General remarks

      • DIRAC 3m
      • WebApp 2m
      • Pilot 2m
      • DIRACOS 2m
      • Documentation 2m
      • OAuth2 2m
      • tornado, M2Crypto, other externals 2m
    • 10:35 → 10:50
      Release planning, tests and certification 15m
    • 10:50 → 11:10
      Weekly development(s) focus 20m
    • 11:10 → 11:20
      DIRAC: current PRs and tasks being worked on 10m

      Ongoing PRs
      - v6r21 PR
      Ongoing tasks
      - ?
      Topics from the google forum
      - ?

    • 11:20 → 11:30
      AOB
      Convener: Federico Stagni (CERN)
    • 11:30 → 11:45
      LHCbDIRAC 15m