BiLD-Dev
Bi-Weekly "Loyal" DIRAC developers meeting. And, following, the LHCbDIRAC developers meeting.
Join Zoom Meeting
https://cern.zoom.us/j/91083694183?pwd=ZkdDY1I1YkJVc2o3UTdBY1BRZE15UT09
Meeting ID: 910 8369 4183
Passcode: 12345678
One tap mobile
+41432107108,,91083694183# Switzerland
+41315280988,,91083694183# Switzerland
Dial by your location
+41 43 210 71 08 Switzerland
+41 31 528 09 88 Switzerland
+41 43 210 70 42 Switzerland
+33 1 8699 5831 France
+33 1 7037 2246 France
+33 1 7037 9729 France
Meeting ID: 910 8369 4183
Find your local number: https://cern.zoom.us/u/artUfAdNB
Join by SIP
91083694183@188.184.89.188
91083694183@188.185.118.153
Join by H.323
188.184.89.188
188.185.118.153
Meeting ID: 910 8369 4183
Passcode: 12345678
BiLD (Bi-weekly DIRAC Development meeting) – 05/08/2021
At CERN: Federico, Chistophe, Christopher, André (in our offices…)
On Zoom: Alexandre, Andrii, Cedric, Hideki, Janusz, Simon, Ueda
Apologies:
Follow-up from previous meetings
- Ran hackathon last week on 7.3.0a17:
- trello board here
- py2 hackathon on lbcertifdirac70 box
- VMDIRAC merged in: the dirac.cfg had to be updated again in puppet
- py3 pilots: some issues with release string parsing, solution merged in
- usual bunch of fixes
DIRAC communities roundtable
GridPP:
Simon
- HTCondor MP tests still ongoing
ILC/CLIC/FCC:
André
- Updated to v7r2
- Some configuration changes not correctly described in the update notes (corrected now)
- VOPolicy (One move incompletely described, one completely missing)
- StatesAccountingAgent (Backend instead of Backends)
- Please consider this when planning to restructure the Operations section (and don’t)
- Some configuration changes not correctly described in the update notes (corrected now)
- Elasticsearch: testing the new opensearch from CERN, seems to work fine with v7r2
LHCb:
Federico+Christophe+Christopher+Alexandre
- py3 is the default client (on CVMFS)
EGI:
Andrii
- NTR
Belle2:
Ueda, Hideki, Cedric
- NTR
DIRAC releases
- v7r2p19:
- Framework
FIX: ComponentSupervisionAgent: consistently use the actual “Setup”, and not the “instance” to look up options. Note: you might need to change your config from “Production” to “DIRAC-Production” depending on your config. - Resources
NEW: removal throttle for Echo
- Framework
- v7r3-pre18:
- (#5035) use DIRACScript instead of Script
- (#5214) Added a HTTPs JobManager, JobMonitor, JobStateUpdate service
- (#5289) SiteDirector: do not send dirac-install if it’s py3 pilot
DIRAC projects
DIRAC:
Issues by milestone:
- v7r1:
- still some old documentation request
- Payload proxy renewal mechanism is screwed when using M2Crypto
- Christophe I will look into it this week
- v7r2:
- Some progress for select.select cannot be used for file descriptors > 1024
- Some progress also for HTCondor related issues.
- v7r3:
Other issues:
- NTR
PRs discussed:
- [v7r2] reuse Operations class for other section
- RFC prepared: https://github.com/DIRACGrid/DIRAC/wiki/approach-to-configuration-navigation-depending-on-VO-and-setups
- Discussed in https://github.com/DIRACGrid/DIRAC/issues/5287
- Andrii I will make another PR, preparing a module/class that then would be used by, e.g., the
Operationsobject. - maybe close the current PR?
- Christophe+Andre’ it would be quite a mess to restructure the /Operations and so better thing very carefully
- might go in a later release, not v7r2 and maybe not v7r3
- RFC prepared: https://github.com/DIRACGrid/DIRAC/wiki/approach-to-configuration-navigation-depending-on-VO-and-setups
- ON HOLD PRs merging strategy
WebApp:
- [v4r3] use default menu if Schema not set
- seems OK
- Desktop theme was removed
- already in previous WebApp pre-release
Pilot:
- Merged
develintomasteron Monday, looks fine
DIRACOS:
- New version v1r24
- Using
davixandSQLAlchemylatest version
- Using
DIRACOS2:
- from previous meeting
fts-reston py3?- NTR
- New release 2.9 not released yet
VMDIRAC:
- Merged. No further developments./
Documentation:
- [integration] remove CommandReference
- tried to make all command documentation created automatically
- Development still ongoing
- Asked to add more documentation on what is actually changing in the PR
- tried to make all command documentation created automatically
- Using sphinx theme
- looks “nice”
- adding a couple of features (e.g. tabs)
OAuth2:
- from previous meeting Any news from issue WLCG and Token transition – reminders and requirements ?
- Today again WLCG meeting
- Nothing new to report
- from previous meeting Andrii forwarded https://docs.google.com/document/d/1ZQzElD866yV6t9AomeW6r-L9s-64pNB95OFDd1EZEbA/edit?ts=60814a09#heading=h.ur0csvxkk8tc which contains technical drawings about the proposed implementation.
- Andrii got access to
puppet, will provide what is needed
tornado/HTTPs
- Added 3 WMS HTTPs services in PR for v7r3
management
- from previous meeting deploying on CVMFS
- Now should be done also for DIRACOS2
diraccfg
- NTR
COMDIRAC
- NTR
DB12
- PR is almost ready, should be finished tomorrow
- pip packages should be deleted, since they are not production
other externals, including Rucio
- PR https://github.com/DIRACGrid/DIRAC/pull/5302
- better way on instantiating the client and running the agents, using DIRAC CS and host certificate
- using the user proxy for running the client
- Already used by GridPP, question if is Belle2 way of working is preserved, Cedric should have a look.
Release planning, tests and certification
-
Certification machines
- lbcertifdirac70 machine:
- py2 installation, for now, but we’ll move to py3 server for the next hackathon
- lbcertifdiracoath machine:
- from previous meeting Andrii seems to be ready. One issue on how to organize Registry.
- client for IAM (approved by Andrea), VO WLCG (not a admin yet) and one client for Check-IN.
- Federico We can run a non-official hackathon here, only for a few people
- from previous meeting Andrii seems to be ready. One issue on how to organize Registry.
- lbcertifdirac70 machine:
-
Next hackathon(s)
- Federico py3 hackathon on lbcertifdirac70
AOB and topics from Google forum
Next hackathon on August 12th: https://indico.cern.ch/event/1065227/
Next BiLD on August 19th: https://indico.cern.ch/event/1065225/
LHCbDIRAC
- https://trello.com/b/BQvEbteg/deploy-v10r2
- ES stuff: we will migrate to OpenSearch, starting from the DIRAC certification, then LHCb certification, and only then Production
- python3 client on CVMFS is now the default for users
- Pilots still using py2
- from previous meetingSandboxStore: installed in prod
- 80% migrated -
rsync-ing
- 80% migrated -
- Tomorrow LHCbDIRAC hackathon based on v7r3/py2