BiLD (Bi-weekly DIRAC Development meeting) – 16/07/2020
At CERN: Nobody, of course!
On Vidyo: Federico, Andrii, André, Christopher, Christophe, Daniela, Hideki, Janusz, Simon, Marko, Aymane, Cedric, Xiaomei, Raja
Apologies: Andrei
Follow-up from previous meeting
DIRAC communities roundtable
GridPP:
- DIRAC release still not on CVMFS? Daniela will contact re-check + contact Andrei
CLIC:
LHCb:
France Grilles:
EGI:
Belle2:
- Migration to v6r22: not yet fully deployed: to be done next week
- TLS issue: https://github.com/DIRACGrid/DIRAC/issues/4675
- Chris can’t fully guarantee that TLS1/2 of pyGSI would work: might try, but it was never tested.
- In the meantime (before moving to v7r0+M2Crypto) they might ask the local security team to make an exception
Nica:
- Error in Pilot related to locale still exists. Some tests were done during work on DIRACGrid/Pilot issue https://github.com/DIRACGrid/Pilot/issues/112
- Strange behavior observed during job submit via JobLaunchpad in WebApp. pilot.cfg added as an argument to any Executable
- [Igor] issue to be opened in the WebAppDIRAC
- Some work togeter with BES3 related to clouds. Somehow amount of pilots on some clouds appear to be more than running VMs. Need further investigation.
Juno:
- Started to do some data challenge test
Current situation
DIRAC
- v6r22:
- v7r0:
- v7r0p28
- Fixes for M2Crypto and FTS3, plus other minor changes
- v7r1:
- v7r1p6
- NEW: (#4641) TransformationCleaningAgent will (re)clean very old transformations that are still in the system
- v7r2:
WebApp:
- A few PRs to review
- The new version of Chrome is not accepting older TLS version - Chris B will make a PR for it
Pilot3:
DIRACOS:
- Created one specific version of DIRACOS “https”, used in the integration test for the tornado PR.
- xroot5? Still to check
- A new version can be created including the newest version of M2Crypto
- master already passing the tests, just need to tag
VM:
- Working with OpenNebula, VMDIRAC working fine but a lot of work operating them.
- Cleaning of old VMs not done - to be checked (see Nica report)
- some bugs found, PRs created, to be reviewed/merged
Documentation:
OAuth2:
-
Big PR: https://github.com/DIRACGrid/DIRAC/pull/4650 from Andrii
- This PR includes the OAuthDIRAC extension.
- The WebAppDIRAC for OAuthDIRAC extension will go in WebApp
- All the tests are passing now: ready for review
- Testing in a development machine, but can’t test it all
- Would need a pre-release and a certification
-
Andrii will prepare a presentation for the next BiLD
-
There might be some overlapping with Chris’s PR about https, which would need to be sorted out
tornado and other externals
-
M2Crypto: main maintainer MIA, the other 2 maintainers started reacting, after Chris contacted them. Chris opened 2 PRs in https://gitlab.com/groups/m2crypto/-/merge_requests and hotfixed in the LHCb setup, and there things work.
- One of the MRs was merged, the other not (the second is less important)
- new tag with it? Anyway it’s in the master branch, for us it’s enough
-
More hotfixes being tested last week, but these fixes are for DIRAC
- prob a PR will be created today
-
tornado: PR made by Chris based on the work done by Luis 2 years ago https://github.com/DIRACGrid/DIRAC/pull/4677
- Some JSON issues are still unresolved and blocking
- the old PR from Luis replaced by this one
-
OAuth PR integrated into this tornado one?
- probably, which means that this tornado PR will be merged first, but some more considerations are needed
management
diraccfg
Release planning, tests and certification
- v7r2 “full”: other newer features can go only for v7r3
Weekly development(s) focus
DIRAC: current PRs and tasks being worked on, or topics from Google forum
PRs:
On issues:
- Nothing not already mentioned above
Topics from Google forum:
AOB
CERN policies still forbid travel. Can’t know yet when there will be next DUW.
Next BiLD in 2 weeks.
LHCbDIRAC
-
BKK
- Cert instance (INT12r) moved to 19c some time ago
- Updates the instant client on the machine (in puppet) [Chris]
- Oracle does not manage package version “normally”
- so, some by hand intervention is needed
- anyway, the old client looks like still working fine
- Will check that a bit more before deploying to production servers
- The prod server hardware has been upgraded July 7th
- The prod server has been upgraded to 19c yesterday
- Federico: I didn’t run the performance tests, only relied on the unit/integration tests
- Chris B: since new version performances “look” better
-
Chris B: we can automatically install LHCbDIRAC in production
- updates in lbtaskrun – already tested and used in the last release deployed
- For the next release there won’t be no need for asking a release to the shifter
- Chris: I would still deploy it for the moment, as old location used by pilots
-
Usage of SDumont restored
-
Chris B: https://gitlab.cern.ch/lhcb-dpa/analysis-productions/LbAnalysisProductions/-/issues/9#note_3624326
- Want to create a new MR for LHCbWebAppDIRAC that would add what’s in the linked screenshot
- Only new things, plan to target the
master branch
-
Chris B: https://indico.cern.ch/event/932570/contributions/3918810/attachments/2074569/3483373/Belle_II_-_HSF_Jul_2020.pdf slide 18 maybe something interesting? (scout jobs, similar to our testing phase in MC jobs, but in WMS)