BiLD-Dev
2-R-014
CERN
Bi-weekly DIRAC Development meeting -- 18/07/2019
At CERN: Federico, Zoltan, Christophe, Andre', Marko, Christopher, Sasha, Wojciech
On Vidyo: Andrei, Andrii, Luisa, Xiaomei, Daniela, Simon, Hideki
Apologies:
Follow-up from previous meeting
- GS DOCS: google evaluating students. We got a few applications and selected 2. News at the end of the month
- Closed several WebApp issues that are now in v4r0
DIRAC communities roundtable
- CLIC:
- moved to v6r21
- LHCb:
- NTR
- CTA:
- moved to v6r20 recently
- new DB for provenance (postgresSQL), using sqlAlchemy for connecting to it. Had to install a python module (could be added in DIRACOS extension when available)
- not registered as a catalog
- GridPP:
- Trying to connect DIRAC to Openstack cloud (questions on the forum)
- Running v6r21p7
- EGI
- Running smoothly, and starting to use OAuth authentication, with some patches directly in production.
- Incorporating new cloud resources, need to incorporate properly Pilot3 in the cloud
- Belle2:
- Moved to v6r21
- Juno:
- deploying production system (TS) and data movement (with TS)
- queries done by the InputDataAgent to the DFC may become very slow (e.g. 800K files returned, way too much)
- not many technical solutions, need to find an operational way around it
- next week planning to upgrade to v6r21
Current situation
DIRAC
- v6r21:
- v6r21p12 is the last patch release
- Minor fixes only in p11 and p12
- v6r21p12 is the last patch release
- v6r22:
- v6r22 and v6r22p1 created
- See the instructions in https://github.com/DIRACGrid/DIRAC/wiki/DIRAC-v6r22 for upgrading
- not many new things
- certified also with DIRACOS
VM:
- NTR. Need to look into using Pilot3
WebApp:
- v4r0p6 is the latest release
- Created branch rel-v4r0
- for v6r22 and v7r0
- integration will go to v4r1 for v7r1 (next)
- JSLint can be added for Travis tests [Zoltan]
- needs node.js
Pilot3:
- Merged devel into master, working fine in production for LHCb
- One PR from Andrew for using MultiLauchAgent has been merged (also in master)
- Wojciech is setting up a test for PilotLogger (with some fixes going into devel branch)
- VM with tornado that sends messages to CERN ActiveMQ instance
- Right now works only with python 2.7, plan to use system calls for python 2.6
DIRACOS:
- Almost done with the script for extending DIRACOS
- started with extending the python packages (this is the need right now)
- v1r3 can be created
Documentation
- NTR
OAuth2:
- Will be used from DIRAC v7r1
DB12:
- NTR
tornado and other externals
- NTR
Release planning, tests and certification
Release planning:
- v6r21 series
- NTR
- v6r22 series
- NTR, only fixes
- v7r0 series
- a rel-v7r0 branch is going to be created including PRs:
- #4160 (autopep8s...)
- #4163 (fixes from last hackathon + Pilot3 is default)
- #4170 (Production system docs and tests)
- which will be for v7r0-pre19
- a rel-v7r0 branch is going to be created including PRs:
- Integration and v7r1:
- PR #4157 (ProxyProviders and OAuth2) will be merged in integration and v7r1-pre1 created with it
- All other PRs can for the moment wait (maybe #4103 and #4108 if updated, and #4166)
Certification process:
- Next week hackathon of v7r0-pre19
Weekly development(s) focus
- DIRACOS extension: how to specify it in releases.cfg?
- decided to use "DIRACOS = LHCb:vXrY" if it comes from an extension
DIRAC: current PRs and tasks being worked on, or topics from Google forum
- Question from Janusz on RSS on the developer forum is answered
AOB
Decided to close the Google developer forum, and simply open github issues instead.
Next BiLD in 2 weeks from now.
---------------------------------------------------------------------------------------------------------------
LHCb
- v9r4
- second hackathon tomorrow
- New bundle needed: include six plus new version of LbEnv
- Needs DIRACOS v1r3 with fixed version of ElasticSearch