Bi-weekly DIRAC Development meeting -- 24/10/2019
At CERN: Federico, Zoltan, Christophe, Alexandre, Andre', Marko, Marcelo
On Vidyo: Daniela, Simon, Janusz, Xiaomei, Andrei
Apologies:
Follow-up from previous meeting
- vyPR: Federico and Andre' met with Josh and his supervisor and said that we might have a look at it again in the future, once we've migrated to tornado for our services
- GOC DB holds little info for non-LHC VOs
- PR for removing SystemLogging created
DIRAC communities roundtable
- CLIC:
- LHCb:
- GridPP:
- FranceGrilles:
- Moved to v7r0, but still sending Pilot2. After 2 weeks of operation quite smooth sailing
- M2Crypto flag off, will be turned on
- Asked Lyon T1 to start a ElasticSearch service
- EGI
- Discussing identity providers (DIRAC - registry part of the CS - might be one of them)
- This discussion will require an architectural decision, after having clearly defined what are the requirements
- Juno+Bes3
- On v6r22 since a while
- working on Transformation system for production system
- asked about ProductionSystem (in v7r0), pointed that it's missing a WebApp
Current situation
DIRAC
- v6r22:
- the last patch contains a backport of the TransformationSystem/DataRecoveryAgent from ILC, and fixes for cleanup of SingularityCEs
- v7r0:
- a tag for v7r0p3 created.
- restored logic for speed up job submission
- added some logic for MP jobs
- v7r1:
- no hackathon last week
- the use of docker for testing, and GitHub actions need some tweaking
- once done, everyone will be invited to try on their own developer installation
VM:
WebApp:
Pilot3:
- pilot logging: tested fully also with python 2.6 and SSL, PR is not yet opened but getting polished
DIRACOS:
Documentation
OAuth2:
tornado and other externals
- JSON encoding PRs: Chris will clarify what to keep and what to close
Release planning, tests and certification
Release planning:
Marko will check (and contact Andrei) on how to use GitHub releases for announcing new releases (no -pre)
- v6r22 series
- few simple PRs ready to be merged
- v7r0 series
- MP fixes
- backporting of docker tests should be cherry-picked
- v7r1 series
- Integration (and v7r2)
- MultiVO DFC PR: need last review from Zoltan
Certification process:
Weekly development(s) focus
Discussion on user jobs and payload environments. Isolation is an issue.
Users tend to work on their payloads within the DIRAC(OS) environment, and this creates issues. Also, users sometime call DIRAC commands within jobs (e.g. for downloading input files)
It's pointed out that this approach just doesn't work (or it's very risky). Isolation of user payloads can be done if using DIRAC workflow package (which uses systemcall) or through containers. Discussion can go on on the forum or within PRs #4300 and #4302
DIRAC: current PRs and tasks being worked on, or topics from Google forum
all discussed above
AOB
Next BiLD in 4 or 5 weeks from now
---------------------------------------------------------------------------------------------------------------
LHCb