BiLD (Bi-weekly DIRAC Development meeting) – 13/08/2020
At CERN: Nobody, of course!
On Zoom: Federico, Alexandre, Andrii, Aymane, Christophe, Christopher, Daniela, Simon, Vladimir
Apologies: André, Marko
Follow-up from previous meeting
After OAuth pres:
- REST interface: RFC for discussion? (see today)
- Not yet started it [Andrei]
- Basic OAuth library:
- Chris Burr started drafting something. "More or less I have a proposal, hope to open an issue before holidays
- RFC for semantic of components?
- not started yet [Andrei/Federico/Andrii]
DIRAC communities roundtable
GridPP:
CLIC:
LHCb:
- Ran 2 more certification hackathons using v7r1. Looks OK, just not much time now for production (which will be in September)
France Grilles:
EGI:
Belle2:
Nica:
Juno:
Current situation
DIRAC
- v6r22:
- v6r22p33
- M2Crypto/pyGSI related fixes
- v7r0:
- v7r0p32
- JobAgent - define DIRAC_JOB_PROCESSORS environment to number of processors allocated for the user job
- WebAppDIRAC v4r0p30
- v7r1:
- v7r2:
WebApp:
- Do update to latest version!
Pilot3:
- Merged one PR for better monitoring of scripts
- One draft PR open for tests of various types of CEs
DIRACOS:
- xroot5?
- ARC6 will need to be fixed first (not for us)
- Marko created one more nightly for HTTPs
less command fixed (not released yet)
VM:
Documentation:
OAuth2:
- Discussed last week.
- Documention is in progress. Only done for now for REST.
tornado and other externals
- tornado: today’s topic
- M2Crypto: …end of line for the fixes?
- still one open PR on M2Crypto side
management
diraccfg
Release planning, tests and certification
- Big PRs for v7r2 are not yet reviewed. Will need to wait for them before really starting certifications. Given the holiday period, we are talking about September.
Weekly development(s) focus
- HTTPs (presentation on the agenda)
- Previous BiLD/presentations/discussions/links also on the agenda
Summary:
- The advantages are clear (use of Standard library, interrogate services without installing DIRAC).
- Services would need to be migrated by inheriting from the new
TornadoService.
- Clients don’t need to be migrated.
- CS
protocol flag needs to change to HTTPs
Dev:
- One route per service, then
POST verb with remote method name and parameters
- One tornado process runs all the services
- Exception for master CS
Issues:
- Input/output must be JSON-compatible (some tasks still open)
- FileTranfer not fully implemeted
- Tornado is not for that. Need to use something else
- Threads are not allowed
- Need to fit into ASync style. Callbacks “poor man” solution.
- Global variables can’t stay (gLogger, gConfig)
- DIRACOS should include:
- patched Tornado 5.1 (fork will come in GitHub/DIRACGrid)
- we need to rely on M2Crypto, for now, and there’s also a patch for it
Discussion:
- Tornado is maybe not what we want. But it’s right now the way to have https protocol. Maybe it won’t stay in the future, maybe it could be replaced one day, but in the meantime DIRAC can start speaking HTTPS
Conclusion:
- Merging this PR should be OK. Running https services need to be manual.
- OAuth PR: what’s done here seems to fit there too.
DIRAC: current PRs and tasks being worked on, or topics from Google forum
PRs:
- v7r0:
- Few PRs ready to be merged, non-problematic
- v7r1:
- One large-ish PR with simple fixes for scripts
- v7r2:
On issues:
Topics from Google forum:
AOB
CERN policies still forbid travel. Can’t know yet when there will be next DUW.
EGI workshop in November will go fully virtual. 1 day DIRAC session? probably - AT in contact.
Next BiLD in 5 weeks.
LHCbDIRAC
- Oracle instantclient to be updated in production too.
There are minutes attached to this event.
Show them.
