BiLD (Bi-weekly DIRAC Development meeting) – 27/07/2023
At CERN:
On Zoom: Federico, Andrei, Bertrand, Cedric, Daniela, Ewoud, Michel, Simon F, Ueda, Hideki, Janusz, Lorenzo, Simon M, Christophe, Christopher, Dhiraj
Apologies:
Follow-up from previous meetings
- Last “standard” BiLD 5 weeks ago
- Trying to catch all updates below
- Last DIRAC certification hackathon on July 6th
- did not do much due to unavailability of Trello for most
- DiracX hackathon on July 4th and 5th: https://indico.cern.ch/event/1292289/
- 9 total attendants
- Mostly worked in pairs, with Christophe running around (while Chris B was in hospital waiting for Olivia! – who was born July 4th, and all are well!)
- Daniela & Simon send their congratulations :-)
- Progressed on several points, but mostly got a first touch of development for DiracX for most of the attendants.
- Documentation partly updated
- BildX meeting on July 29th
- did not go through all the bullets in the uploaded slides
- recording added to the agenda
DIRAC communities roundtable
LHCb:
Federico+Simon+Christophe+Christopher
- Running on latest v8.0 and latest DIRACOS2 versions
- Using tokens for submitting to all HTCondor CEs
- IAM for LHCb set up, but effectively only for the above purpose
- Using AREX CE instead of ARC/6
EGI
Andrei+Bertrand
Juno
Andrei
- Updated to 8.0.24 – rather smooth
- Might duplicate installation - 1 for Juno and a second for other VOs aswell
- Own IAM isntance – pilot submission with tokens set up.
CLIC/ILC/Calice
André
- Migrated to v7r3 (py3)
- Setting up v8 testing (Lorenzo)
Belle2
Hideki, Michel, Ueda
- v7r3 py2
- DIRACOS2 migrated to py3.11, showed incompatibility with one of the external services.
- openssl handshake failure – will open an issue, maybe there’s a workaround
GridPP:
Daniela, Simon, Janusz
- Running 8.0.23 in production due to ARC meltdown in 8.0.24 (addressed here: #7095 and here: #7104 )
- Advising users to install diracos 2.33 and 8.0.21 to avoid the confusing cfg file not found issue: #7088 and mangled prompt: https://github.com/DIRACGrid/DIRACOS2/issues/109 - while technically harmless, we prefer our users to report real errors.
- Federico DIRACOS2 issue was solved within version 2.35
- Added TransformationSystem to production server. Simon not happy with security model, being addressed here: https://github.com/DIRACGrid/DIRAC/pull/7113 (which now has also spawned Federico’s #7124 to address userDN -> username transition)
- Trying to configure one of our preprod instances (https://diracdev.grid.hep.ph.ic.ac.uk:8443/DIRAC/ visible only if you are a member of the gridpp VO) to use tokens. It’s not going well: #7123 (also: #7126)
- Andrei Maybe a temporary solution is retrying with certificates
- We managed to get one job for one VO through using a token for the pilot late yesterday afternoon. However the token tag seems to be CE specific. For us there is no guarantee that all VOs on a CE will be token ready at the same time. Is there a plan to handle this ?
It would really help if we had a working example (certification server ? – then at least all the admins could see it) somewhere, including a description of a working test. Any help is appreciated. - Related: RAL just had an IAM hackathon which Simon and me attended, and we were hoping to have this up an running by then, but this unfortunately failed. The IAM developers are now however aware of our multi-VO everywhere dilemmas, so there might be an opening to get some multi-VO changes through.
Topics from GitHub/Discussions
only un-answered topics with discussion updates:
DIRAC releases
DIRAC projects
DIRAC:
Issues by milestone:
v8.0:
v8.1:
- 15+ open issues
- Federico Closed some of them as not going to be implemented anymore (DiracX taking over)
Other issues:
PRs discussed:
- For Janusz & Federico: Can we have a quick check how far off the remote pilot logging is from release ? #6208
- Federico merged yesteday, ready for being tested in next week’s DIRAC certification hacakthon
WebApp:
Pilot:
DIRACOS:
- Few fixes/workaround, created version 2.35 2 days ago
Documentation:
OAuth2:
- from previous meeting
- Andrei request from EGI to demonstrate that one VO can run with tokens only
- Check In is progressing:
compute scopes available, they are accepting the idea of using client access tokens (possibility to associate a client to a given VO). They would probably not accept a same client to deal both with client and user access tokens (security concerns with the scopes available in the clients). - WLCG timeline document: https://zenodo.org/record/7014668#.YyLag9JBwQ9
- Still pending the test for ARC7 and CheckIN tokens
- Multiple clients per IdP might be effectively needed
tornado/HTTPs
management
- from previous meeting 3 issues left, still valid?
diraccfg
COMDIRAC
Daniela
- Abandoned until we fix tokens and TransformationSystem.
DB12
Alexandre
- Ewoud benchmarking LHCb sites with DB12 and HEPScore in order to compare the 2.
- Andrei what would be the future of DB12? Keep adding factors?
- Christopher Python itself is getting faster, so that’s going to be a factor to consider. Depending from that is not ideal.
Rucio
Tests
DiracX
- Called a 2nd DiracX hackathon: https://indico.cern.ch/event/1304626/ for 4-5 September. Again at CERN, please do register if you plan to attend.
- Also called a BildX meeting for August 31st (the week before the hackathon)
Release planning, tests and certification
Certification machines
- lbcertifdirac70 machine:
- from previous meeting Federico not rush, but should we move to a Alma9 box?
- Outside of CERN would be better, in CC probably
- Andrei machine is already there, need to decide how to set this one up
- We could also use the new box to test the installation procedure
Next hackathon(s)
- in 1 week, “standard” v8.1.0aX one.
AOB
Next hackathon: On August 3rd
Next BiLD: On August 10th
Half-published the timetable of the DIRAC and Rucio WS: https://indico.cern.ch/event/1252369/timetable/#20231016.detailed
LHCbDIRAC