Choose timezone
Your profile timezone:
BiLD-Dev
→
Europe/Zurich
Description
Bi-Weekly "Loyal" DIRAC developers meeting. And, following, the LHCbDIRAC developers meeting.
Join Zoom Meeting
https://cern.zoom.us/j/91083694183?pwd=ZkdDY1I1YkJVc2o3UTdBY1BRZE15UT09
Meeting ID: 910 8369 4183
Passcode: 12345678
One tap mobile
+41432107108,,91083694183# Switzerland
+41315280988,,91083694183# Switzerland
Dial by your location
+41 43 210 71 08 Switzerland
+41 31 528 09 88 Switzerland
+41 43 210 70 42 Switzerland
+33 1 8699 5831 France
+33 1 7037 2246 France
+33 1 7037 9729 France
Meeting ID: 910 8369 4183
Find your local number: https://cern.zoom.us/u/artUfAdNB
Join by SIP
91083694183@188.184.89.188
91083694183@188.185.118.153
Join by H.323
188.184.89.188
188.185.118.153
Meeting ID: 910 8369 4183
Passcode: 12345678
BiLD (Bi-weekly DIRAC Development meeting) – 21/04/2022
At CERN (in our offices) Federico, Alexandre, Christopher
On Zoom: Andrei, Cedric, Daniela, Janusz, Simon, Hideki, Michel Hernandez Villanueva, Ruben, Xiaomei
Apologies: André
Follow-up from previous meetings
- hackathon of April 7th, on 8.0.0a19: https://trello.com/b/E1OeA7TP/v800a19
- only “minor” issues
- https://trello.com/c/adJ8OZqg/30-verify-wms
- Disappearing of the Framework/Monitoring in 8.0: do we need to prepare it in v7r3?
- maybe not
DIRAC communities roundtable
GridPP:
Daniela+Simon+Janusz
- Nothing to report (it was Easter after all). We are still running v7r2p43. If possible we’d like https://github.com/DIRACGrid/DIRAC/issues/5968 (UTF-8 handling in py2) resolved as soon as possible - we’ve just made a pull request - , as it keeps tripping us up. Might be one for Chris B to look at.
- Christopher probably OK
LHCb:
Federico+Christopher+Alexandre
- NTR
EGI:
Andrei
- Running 7.3.15 (waiting for “approval”)
- One community needing uDocker: would it be possible to provision it in DIRACOS2? It’s a
pip installable- Christopher probably possible
- Still has not understood how to add extensions
ILC/CLIC/FCC:
André(not connected)
- Added CTA to
Systems/StorageManagement/Production/Agents/StageMonitorAgent/StoragePlugins, staging works great
Belle2
Hideki
- Taking time to sort the incompatibility of the extension with v7r2. Deploy after the DUW
- Preparing ES and MQ for Belle2
- deployed ES 7.10
BES3
Xiaomei
- Working on migration to python3. In production still running v7r1.
Topics from GitHub/Discussions or Google forum
- Problems with CI tests
- sorted out.
- Proper way to make a cache persitent in DIRAC
- Andrei maybe try (or get inspired) by what is done in COMDIRAC.
- from previous meeting diracos2/libcloud/conda forge
- Chris updated libcloud on conda forge
- Rucio developers updated rucio on conda forge
DIRAC releases
- v7r2:
- NTR
- v7.3:
- No new releases, but some commits merged in.
- 8.0.0a19:
- No new releases, but some commits merged in.
DIRAC projects
DIRAC:
Issues by milestone:
- v7r2:
- NTR
- v7r3:
- CloudDirector -> SiteDirector
- Reaching a solution, but won’t remove (yet) the CloudDirector
- CloudDirector -> SiteDirector
- v8.0:
- NTR?
- v8.1:
- NTR?
Other issues:
- Not discussed
PRs discussed:
- [7.3 and up] Introduce a CloudComputingElement:
- Still not merged, but PR approved by Andrei
- Should be enough to replace the CloudDirector
libcloudusage has the potential to largely simplify part of the https://github.com/DIRACGrid/DIRAC/tree/integration/src/DIRAC/Resources/Cloud code (some can simply be dropped)- To be verified: EC2 and OpenNebula support
- [integration] Reorganizing monitoring flags
- proposal to simplify drastically the monitoring flags. No discussion, accepted.
WebApp:
- NTR
Pilot:
- Janusz ditched the Extended Logger, not made a PR yet. Quite some code can be made simpler, both on client and server. No dictionary building on the client side.
- from previous meeting
- Janusz server PR restructured a bit, e.g. using standard DIRAC loading mechanism. The ElasticSearch backend has not been added yet (only the file backend)
- Janusz client: the current system of RemoteLogging is too complex. The suggestion from Chris Burr to capure the outputs in the pilot wrapper look much simpler, investigating that and trying different options.
- Federico remember that the PilotWrapper created in DIRAC is only used by the SiteDirectors, e.g. not by Cloud yet. This might change with the new PR #5988 (to check)
DIRACOS2:
- GridPP: Is there a release schedule for DIRACOS2 ? (both libcloud (thanks, Chris) and rucio-client have been updated in conda-forge)
Documentation:
- from previous meeting diracgrid.org still forwards to http://mardirac.in2p3.fr
OAuth2:
- from previous meeting Any news from issue WLCG and Token transition – reminders and requirements ?
- NTR
tornado/HTTPs
- from previous meeting Andrii We can eventually have special classes in Configuration and BundleDelivery to pass binary data - to be discussed.
management
- NTR
diraccfg
- NTR
COMDIRAC
- NTR
DB12
- NTR
Rucio
- from previous meeting PR merged, what’s next?
- Janusz we have to find some users, will need to write instructions on how to use it
Release planning, tests and certification
-
8.0 release
- Currently missing:
- a few critical issues, but for OAuthZ we don’t want to improve ATM
BLOB->TEXTeverywhere (only 1 DB missing)
- Not in a hurry to have the release out. Target: 3/4 more weeks.
- Currently missing:
-
Certification machines
- lbcertifdirac70 machine:
- NTR
- lbcertifdiracoauth machine:
- NTR
- lbcertifdirac70 machine:
-
Next hackathon(s)
- Still on lbcertifdirac70
AOB
Next hackathon on April 28th
Next BiLD on May 5th
from previous meeting consortium agreement: will go through another round. Still in the phase of sinchronizing the text.
from previous meeting DIRAC Workshop. 9-10 May 2022. Will stay virtual, and the presentations will stay in the mornings (similar schedule to last years’).
LHCbDIRAC
- v10r3: deploy board in https://trello.com/b/kzUKdMts/deploy-v10r3
- lbwms: considerations in https://codimd.web.cern.ch/5-ImZHjFTi6ko_a-5-5BFw#
- LHCbDIRAC hackathon based on 8.0
- can start looking into it, first week of May
- Started to use MareNostrum:
- 200 jobs running in parallel (can probably go up to ~250)
- currently fixing one of the last “major” issue (I hope) related to accounting reports
- 50kh available: 84% has been consumed in a month
- Subset of CVMFS containing Gauss is updated automatically if required (no issue so far)
- Summary: the solution is still far from being perfect but works.
There are minutes attached to this event.
Show them.