BiLD-Dev
Bi-Weekly "Loyal" DIRAC developers meeting. And, following, the LHCbDIRAC developers meeting.
Join Zoom Meeting
https://cern.zoom.us/j/91083694183?pwd=ZkdDY1I1YkJVc2o3UTdBY1BRZE15UT09
Meeting ID: 910 8369 4183
Passcode: 12345678
One tap mobile
+41432107108,,91083694183# Switzerland
+41315280988,,91083694183# Switzerland
Dial by your location
+41 43 210 71 08 Switzerland
+41 31 528 09 88 Switzerland
+41 43 210 70 42 Switzerland
+33 1 8699 5831 France
+33 1 7037 2246 France
+33 1 7037 9729 France
Meeting ID: 910 8369 4183
Find your local number: https://cern.zoom.us/u/artUfAdNB
Join by SIP
91083694183@188.184.89.188
91083694183@188.185.118.153
Join by H.323
188.184.89.188
188.185.118.153
Meeting ID: 910 8369 4183
Passcode: 12345678
BiLD (Bi-weekly DIRAC Development meeting) – 11/03/2021
At CERN: Still, nobody :/
On Zoom: Federico, Alexandre, André, Andrei, Andrii, Cedric, Christophe, Christopher, Daniela, Hideki, Janusz, Marko, Simon, Vladimir, Xiaomei
Apologies:
Follow-up from previous meeting
-
Ran hackathon last week on v7r2-pre37:
- trello board here
- VMDIRAC not yet fully installed nor tried
- ran with CS on https, minor fixes had to be done (mostly doc). Also to
puppetmodule- v7r1 clients are not compatible with installations where the CS runs on https, but it was expected
- Some more fixes had to be done
-
- André No time to investigate yet.
- Andrei I made some remarks, I’d like an update. If we are switching to this mode, we never come back.
- Christopher maybe not too bad. We can go back to old mode if we don’t like it.
- Andrei we can’t anyway keep both in parallel, it’s one or the other.
- “Random” developers need special attention.
DIRAC communities roundtable
GridPP:
Daniela
- NTR
CLIC:
André
- Scheduling update to v7r0
LHCb:
Federico+Chris+Chris+Marko
- We ran yesterday a LHCbDIRAC hackathon with v7r2-pre38. Quite OK, a few PRs with minor fixes created.
- Managed to run a few services on a kubernetes cluster.
- Using CTA at CERN (replacement of Castor)
- CTA will also be used at RAL
EGI:
Andrei
- Still preparing the migration.
France Grilles:
Andrei
- Started running some SiteDirectors with Pilot3
Belle2
Hideki
- NTR
BES3/Juno:
Xiaomei
- Still preparing update to v7r0.
DIRAC releases
- v7r1p31 + v7r1p32:
- CHANGE: (#5001) JobAgent: if we don’t match a job, independently from the reason, we wait a bit longer before trying again
- CHANGE: (#5001) JobAgent: moved check of availability of CE before doing anything else
- NEW: (#4998) JobWrapper - find executables from PATH if not using an absolute filename
- v7r2-pre37 + v7r2-pre38:
- CHANGE: (#4997) Use prompt-toolkit for password input when generating proxies
- CHANGE: (#4994) dirac-install.py should now be taken from management
- CHANGE: (#5010) Use ConfigurationClient in CSCLI and CSShellCLI
- CHANGE: (#5026) Remove DIRAC.Core.Utilities.ExitCallback
- CHANGE: (#5019) Remove IRODSStorageElementHandler
DIRAC projects
DIRAC:
By milestone:
- v7r1:
- almost only documentation is still there to be done
- v7r2:
- non-urgent issues
- v7r3:
- Make RSS VO-aware: PR created against v7r1, will need to be ported to v7r3
- comments in there…
- OAuth2 will be the main feature (see below)
- Make RSS VO-aware: PR created against v7r1, will need to be ported to v7r3
WebApp:
nginxconfiguration here: https://github.com/DIRACGrid/DIRAC/wiki/DIRAC-v7r2#web-configuration- what to do? Just re-phrase it, make sure it’s clear that it is optional.
- Merged a few PRs for v4r2 (SiteSummary page)
- Discussion in https://github.com/DIRACGrid/WebAppDIRAC/issues/427 for Belle2 WebApp application is ongoing. Should not be difficult to fix.
Pilot3:
- Updates for Jenkins (CC7 default, and CS using http) and
dirac-installfollowing the version in DIRACGrid/management
DIRACOS:
- NTR
DIRACOS2:
fts-reston py3?- not yet (soon?)
- DIRACOS2 builds for ARM and Power8 (compatible with Power9?) are there. They have been also tested.
- Question for
rucioclient… (issue)? (Cedric to answer) - Review from Andre’?
- consider it “done”.
- first non-pre-release today.
VMDIRAC:
- NTR
- Almost got it working in Certif server.
Documentation:
-
PR [v7r2] Use a conda environment when building the docs
- biggest problem is in a feature of rtd. Migrating from
piptocondaon the main repo should be enough to fix it. We should just create v7r2 and changemasterto it.
- biggest problem is in a feature of rtd. Migrating from
-
Federico I have installed DIRAC for local devs using
condas explained in https://dirac.readthedocs.io/en/integration/DeveloperGuide/DevelopmentEnvironment/DeveloperInstallation/editingCode.html- everything was OK (minor:
conda/mambaare not inPATHby default)
- everything was OK (minor:
OAuth2:
- The existing PR will be rebased ASAP (new PR actually will be done.)
- News from issue WLCG and Token transition – reminders and requirements ?
- Trying to follow all discussions, not yet conclusive
- Is it time for a general discussion? Maybe a brainstorming is needed.
tornado/HTTPs
- NTR (just small PR).
management
- dirac-install and globalDefaults uploaded to CVMFS
- merged
- 2 tasks left for deploying on CVMFS
- Marko should come from DIRAC main repo.
diraccfg
- NTR
other externals, including Rucio
- Rucio File Catalog implementation from Belle
- Cedric not connected today…
- Needs (urgently?) a working implementation
- GridPP has not yet tested it, but some comments went in the PR, especially on the point of configuration.
- the GridPP Rucio server is multiVO, and not fully working yet.
Release planning, tests and certification
-
Certification machine
- VMDIRAC
puppetupdated for VMDIRAC. Please don’t updateetc/dirac.cfgorbashrcby hand.puppetruns every 30 mins.- nearly there
- Reminder: task for multi-vo
- Daniela I will write some multi-vo tests
- VMDIRAC
-
Next hackathon
- next week (18th March)
- first hackathon for v7r3-pre1
- next week (18th March)
Weekly development(s) focus
upgrading to v7r2
- The structure with
src/PROJECTNAME/code...should be adopted - A
setup.pyshould be written - For Vanilla projects this affects COMDIRAC
- VMDIRAC and WebAppDIRAC are only server, so no need to update immediately.
- in LHCb we:
- created a
setup.py, and we can dopip install LHCbDIRAC - updated quite some code for python3
- created a
AOB and topics from Google forum
Next hackathon on March 18th: https://indico.cern.ch/event/1015938/
Next BiLD on March 25th: https://indico.cern.ch/event/1015936/
DUW10 online: https://indico.cern.ch/e/DUW10 10-11th May 2021:
- announced “officially”, first registrants (not you!)
- Sent a mail to presenters…
LHCbDIRAC
- v10r2 planning in https://gitlab.cern.ch/lhcb-dirac/LHCbDIRAC/-/wikis/Releases-planning#based-on-dirac-v7r2
- https://codimd.web.cern.ch/2es8HPUcSmS42jrTxhX3JQ?both
- MySQL 8: ticket open for upgrade. Now we can upgrade
- ES7:
- no centralized logging
- To be done
- https://gitlab.cern.ch/-/snippets/1547