Choose timezone
Your profile timezone:
BiLD-Dev
→
Europe/Zurich
Description
Bi-Weekly "Loyal" DIRAC developers meeting. And, following, the LHCbDIRAC developers meeting.
Join Zoom Meeting
https://cern.zoom.us/j/91083694183?pwd=ZkdDY1I1YkJVc2o3UTdBY1BRZE15UT09
Meeting ID: 910 8369 4183
Passcode: 12345678
One tap mobile
+41432107108,,91083694183# Switzerland
+41315280988,,91083694183# Switzerland
Dial by your location
+41 43 210 71 08 Switzerland
+41 31 528 09 88 Switzerland
+41 43 210 70 42 Switzerland
+33 1 8699 5831 France
+33 1 7037 2246 France
+33 1 7037 9729 France
Meeting ID: 910 8369 4183
Find your local number: https://cern.zoom.us/u/artUfAdNB
Join by SIP
91083694183@188.184.89.188
91083694183@188.185.118.153
Join by H.323
188.184.89.188
188.185.118.153
Meeting ID: 910 8369 4183
Passcode: 12345678
BiLD (Bi-weekly DIRAC Development meeting) – 11/11/2021
At CERN: Federico, Alexandre, Christopher (in our offices…)
On Zoom: Andrei, Andrii, Hideki, Janusz, Simon, Xiaomei, Ueda, Igor, Alex Owen
Apologies: André
Follow-up from previous meetings
- Andrei New RFC/Discussion to replace ?
- NTR
- hackathon of 4th November, 8.0.0a7:
- rather smooth, but TBH not much new to test
- for the next one we would really need to decide what to do
DIRAC communities roundtable
GridPP:
Daniela+Simon+Janusz+Alex
- Almost ready to deploy v7r3. Moving directly from v7r1 to v7r3.
- Alex Owen is new member of GridPP. Will focus on the TransformationSystem.
ILC/CLIC/FCC:
! André
- NTR
LHCb:
Federico+Christophe+Christopher+Alexandre
- Will move to v7r3 in production (first py2, then py3 – already tested on certification setup) next week
EGI:
Andrei
- Running servers at v7r3p8, python2 pilots.
- Stepping up using clouds with VMDIRAC (v7r3p8, python3 pilots). Some streamlining of the VMs configuration is needed - discussing with Simon.
- using LocalCEType=Pool+InProcess (not yet Pool+Singularity)
Belle2:
Hideki
- Migration to v7r1: hopefully soon
- Questions about py3 migration for DIRAC extensions
- Reminders:
- Main question was about publishing on pypi. As of now, it’s been considered the norm, but there could be cases when this is not possible. The DIRAC Pilot might need to be slightly modified if this is the case.
- Christopher ideas are there, but requirements are not felt yet. If there would be, I can share my ideas.
BES:
Xiaomei
- Similar requirements to Belle2 for the migration to py3
- See also question on the forum.
NICA:
Igor
- in the last 2 months there were no particular updates.
- For py3 server migration: how?
- Federico See
bashrcin https://github.com/DIRACGrid/DIRAC/wiki/DIRAC-7.3-(v7r3), and do NOT usedirac-installwhich is only for py2. Use the systemAdministrator instead.
- Federico See
Topics from GitHub/Discussions or Google forum
- from previous meeting Christophe I suggest to close the google forum
- Federico kind of agree, but needs to update some links
- Daniela Requires users to actively set “watch all discussion” to emulate google forum behaviour; how do we avoid losing the people who have so far contributed in the google forum; higher barrier to entry/makes it look like it’s for “experts” only - I think google groups + issues for anything that is actually identified to be an issue was a much better setup.
- Andrei the github discussions is effectively for developers
- ==> so, leave them both
DIRAC releases
- v7r2p33:
- WorkloadManagement (#5553) Various fixes to executor infrastructure
- ConfigurationSystem (#5524) VOMS2CSAgent: VOMS users that have multiple DNs and are suspended should no longer lead to “User not registered” errors and have their status correctly set
- Core (#5474) Always get voms-proxy-init from DIRACOS to avoid issues generating proxies on CentOS 8
- 7.3.8:
- WorkloadManagementSystem (#5564) JobParameters - generalize getting CS parameters for Cloud VMTypes also
- 8.0.0a8:
- NTR
DIRAC projects
DIRAC:
Issues by milestone:
- v7r1:
- only 2 issues still open, for documentation, for Andrei
- v7r2:
- ARC CE improvements: Raja created draft PR with a new CE type “AREX”
- Jobs cannot be removed by JobCleaningAgent if user DN changes : this is actually becoming an increasing problem in LHCb, but there’s no easy solution. Federico will try to take care ASAP
- v7r3:
- Not much changes since last time
- v8.0:
- Not much changes since last time
Other issues:
- Nothing discussed
PRs discussed:
- Went through some of the PRs
WebApp:
- CI action updated, not yet fully tried out
Pilot:
- NTR
DIRACOS:
- from previous meeting we’ll use the forked elastcsearch open clients from https://github.com/orgs/opensearch-project/repositories:
- Federico I will take care of updating our code
- only add
opensearchclients to DIRACOS2
DIRACOS2:
- from previous meeting
fts-reston py3?- fts3-rest is added
- the “official” client will be worked “soon”
- PR finally merged
- from previous meeting few issue with
gfal2andgsoap, working to put in condaforge- soon it should be available
- Christophe
gsoapis only for SRM, maybe it won’t be needed…? Only used for tape. I will get news today about it
ruciois there
Documentation:
- from previous meeting André: diracgrid.org forwards to readthedocs. Where can the consortium agreement etc. now be found?
- Andrei I have to restore it
OAuth2:
- from previous meeting Any news from issue WLCG and Token transition – reminders and requirements ?
- pre-GDB at the beginning of the week
- WLCG profile will be updated, merged with sciToken.
- new RFC about access token profile
- Andrii WLCG token transition timeline: https://docs.google.com/document/d/11fcZU8fEsfjDiSkjh95nVr4tNXLPCA_xwr2SwriBpiw/edit#heading=h.s0j7quda1urv
- from previous meeting Need to re-take on board some old discussions, e.g. https://demo.hedgedoc.org/38mzdSQpT36MifxBp7xX8g
- Proposal to “move Registry to IdPs” (Andrii showed a few slides regarding it)
- e.g. DIRAC Groups == VOMS roles
- presented proposal to make a “DIRAC Registry service” that groups information from different IdPs. The CS/regsitry section could be considered as an IdP
- Andrii presentation:
https://indico.in2p3.fr/event/25381/contributions/100970/attachments/66690/93333/Dirac authentication_authorization system.pptx - Federico a pre-requisite is stop using DNs: https://github.com/DIRACGrid/DIRAC/issues/4484
- Ueda you mentioned only VOMS Roles, what about Groups?
- Andrei normally, we recommend not to use groups. We would probably give similar suggestions for IaM. Anyway, will keep mapping to DIRAC groups for describing the access to DIRAC functionalities.
- discussion went on for a little while with clarifications on future interactions.
tornado/HTTPs
- NTR
management
- DIRACOS2 update to go on CVMFS? (issue is there, but stalling)
- Andrei for the moment there’s no automatic update. Should check if the cron job is taking care.
diraccfg
- NTR
COMDIRAC
- moving to py3
DB12
Alexandre
- from previous meetings Port to python3:
- next is (i) merge the PR (ii) make the package (iii) use it in DIRAC (iv) apply the “fix” 1.16 - 1.19.
- Imane’s PR has been merged, I created another PR based on her work to correct the score according to the python version and the CPU architecture used: https://github.com/DIRACGrid/DB12/pull/5
- merged
- Analysis of the scores hosted on Zenodo: https://zenodo.org/record/5647834
other externals, including Rucio
- 2 PRs in rucio have been approved. Still some issues being solved, will be discussed today in Rucio dev. If OK on rucio side, the current draft PR will be ready for review.
Release planning, tests and certification
-
Certification machines
- lbcertifdirac70 machine:
- NTR
- lbcertifdiracoauth machine:
- NTR
- lbcertifdirac70 machine:
-
Next hackathon(s)
- next week
AOB
Next hackathon on November 18th
Next BiLD on November 25th
LHCbDIRAC
- TECH started: https://codimd.web.cern.ch/0VG3xjEWR3KXnlWT8osSYA?both
- Submit project for ISIMA student
- project has been uploaded to the platform
- v10r3: deploy board
There are minutes attached to this event.
Show them.