BiLD-Dev
Bi-Weekly "Loyal" DIRAC developers meeting. And, following, the LHCbDIRAC developers meeting.
Join Zoom Meeting
https://cern.zoom.us/j/91083694183?pwd=ZkdDY1I1YkJVc2o3UTdBY1BRZE15UT09
Meeting ID: 910 8369 4183
Passcode: 12345678
One tap mobile
+41432107108,,91083694183# Switzerland
+41315280988,,91083694183# Switzerland
Dial by your location
+41 43 210 71 08 Switzerland
+41 31 528 09 88 Switzerland
+41 43 210 70 42 Switzerland
+33 1 8699 5831 France
+33 1 7037 2246 France
+33 1 7037 9729 France
Meeting ID: 910 8369 4183
Find your local number: https://cern.zoom.us/u/artUfAdNB
Join by SIP
91083694183@188.184.89.188
91083694183@188.185.118.153
Join by H.323
188.184.89.188
188.185.118.153
Meeting ID: 910 8369 4183
Passcode: 12345678
BiLD (Bi-weekly DIRAC Development meeting) – 06/05/2021
At CERN: Still, nobody :/
On Zoom: Federico, Alexandre, André, Andrei, Andrii, Cedric, Christophe, Christopher, Daniela, Hideki, Janusz, Marko, Simon, Xiaomei
Apologies:
Follow-up from previous meeting
- Ran hackathon last week on v7r3-pre7:
- trello board here
- VMDIRAC: tag was not there, so the latest version was not installed.
- Q: who creates the tags?
- Simon will create the tags
- Q: who creates the tags?
- ComponentMonitoring still looks problematic.
- PRs merging strategy?
- Marko Working on GitHub Action – should be finished tomorrow
- prob only applying from v7r2
- Marko Working on GitHub Action – should be finished tomorrow
DIRAC communities roundtable
GridPP:
Daniela
- Overload of the file catalog follow-up?
- haven’t seen it again (no changes done)
- NeTR
CLIC:
André
- NTR
LHCb:
Federico+Chris+Chris+Marko
- Nothing interesting to report
EGI (+ France Grilles):
Andrei
- M2Crypto and VOMS extension follow-up (issue)
- operationally solved for the moment
- reproducer is needed for trying to fix the issue – will be added to issue
- problem with retrieving the output for ARC
- v7r1 is using DIRACOS v1r18 so no
arc6(which is in v7r2) - will try to install it with
--dirac-os-version=v1r21
- v7r1 is using DIRACOS v1r18 so no
- Moved all components from cyfronet, migration is over
Belle2
Hideki
- v7r0 migration – ongoing
- updated DIRACOS to v1r18 (in v6r22)
webdavnow working fine
BES3+JUNO
Xiaomei
- NTR
DIRAC releases
- v7r1p39:
- (#5121) for HTCondor, the SiteDirectory write the executable in the globally defined working directory
- *v7r2p6:
- (#5102) JobCleaningAgent will first DELETE and only then REMOVE jobs
- *v7r3-pre7:
- (#5110) Glue2 is now the only way
- (#5000) Initial support for Python 3 server installations
DIRAC projects
DIRAC:
Issues by milestone:
- v7r1:
- Some recently added issues
- Some old ones for documentation still to finish
- v7r2:
- NTR
- v7r3:
- NTR
PRs discussed:
- [v7r1] Add sanity check for reading CA dir
- Christophe do we need this check at all?
- Daniela I think it might be useful, but it’s a corner case. In theory it should never happen
- finally, remove the check
- [v7r1] Add multi-node allocation support in the CEs (srun with LocalCE and SSHCE)
- for HPC (for executing multiple pilots in parallel)
- Logic looks fine, but needs another review
- [v7r2] Make DIRAC.Core.Base.Client.Client threadsafe
- Looks OK but needs review
- [v7r3] register args (optional)
- this is OK and can be merged. [v7r3] move Script goods to DIRACScript(wait #5061) will need the above merged and some more refactoring after that
WebApp:
- Preparing for pip package: created release v4r3-pre3
- ToDo:
- change for adapating to new structure
- upload to pypi
- ToDo:
- couple fixes after hackathon merged and released
Pilot3:
- NTR
DIRACOS:
xroot5?- NTR
arc6- Looks fine: in v1r21 (from v7r2p6)
condor 8.8?- NTR
DIRACOS2:
- from previous meeting
fts-reston py3?- NTR
- from previous meeting Question for
rucioclient… (issue)?- Not done yet (not fully in our hands)
- NeTR
VMDIRAC:
- NTR
Documentation:
- NTR
OAuth2:
-
from previous meeting Any news from issue WLCG and Token transition – reminders and requirements ?
- Recently had a discussion with IaM developer (Andrea). Mostly technical.
-
We created a second certification setup where to test these OAuth interactions: CC7 box is there, as well as MySQL 8: started installing DIRAC there. The
dteamIaM instance will be added there.- it was discussed if to add also the LHCb IaM but we decided against it
- we should probably need admin access to dteam IaM – probably also managed by Andrea
- Andrei+Andrii will ask about that
-
Andrii forwarded https://docs.google.com/document/d/1ZQzElD866yV6t9AomeW6r-L9s-64pNB95OFDd1EZEbA/edit?ts=60814a09#heading=h.ur0csvxkk8tc which contains technical drawings about the proposed implementation.
- anyone can comment on it as now. Please do read it!
-
Andrii I will split the existing PR for making it easier to comment/understand
tornado/HTTPs
- NTR
management
- from previous meeting 1 task left for deploying on CVMFS
- Not yet done
- Add same check to VOMS_DIR and VOMSES as there is for CERT
- Asked to add the created
bashrctoqabranch of puppet modules
- Asked to add the created
- Q: Test on CentOS8? (for DIRACOS2/python3)
- not really any need to do them (being conda-based it should be OK)
COMDIRAC
- NTR
other externals, including Rucio
- the Rucio client as of now requires a configuration file
- changes there does not depend from us: issue has been opened and it’s being looked into.
- PR: https://github.com/DIRACGrid/DIRAC/pull/5067
- failing tests
- no updates since last time
- Cedric I would like to test the changes better that now. E.g. using github actions, or in a certification setup
- Federico For the moment (this PR is for v7r0) just make it to pass the existing tests, and we can merge it. Once merged, the integration tests run by GitHub Actions can be added to release branch v7r2.
Release planning, tests and certification
-
Certification machine(s)
- created wiki: https://github.com/DIRACGrid/DIRAC/wiki/Certifications where there are some reminders + monitoring links for the 2 certification machines.
-
Next hackathon
- in 2 weeks (20th May)
- “regular” v7r3-preX
- in 2 weeks (20th May)
Weekly development(s) focus
NTR
AOB and topics from Google forum
- DUW next week
- Timetable is ready: https://indico.cern.ch/event/852597/timetable/#all.detailed
Next hackathon on May 20th: https://indico.cern.ch/event/1037152
Next BiLD on June 03rd
LHCbDIRAC
- v10r2 planning in https://gitlab.cern.ch/lhcb-dirac/LHCbDIRAC/-/wikis/Releases-planning#based-on-dirac-v7r2
- https://codimd.web.cern.ch/2es8HPUcSmS42jrTxhX3JQ?both
- No news
- At some points we should install a python3 client on CVMFS
- in the hands of lb-dirac (then LHCbPilot would need to add a flag)