BiLD-Dev
Bi-Weekly "Loyal" DIRAC developers meeting. And, following, the LHCbDIRAC developers meeting.
Join Zoom Meeting
https://cern.zoom.us/j/91083694183?pwd=ZkdDY1I1YkJVc2o3UTdBY1BRZE15UT09
Meeting ID: 910 8369 4183
Passcode: 12345678
One tap mobile
+41432107108,,91083694183# Switzerland
+41315280988,,91083694183# Switzerland
Dial by your location
+41 43 210 71 08 Switzerland
+41 31 528 09 88 Switzerland
+41 43 210 70 42 Switzerland
+33 1 8699 5831 France
+33 1 7037 2246 France
+33 1 7037 9729 France
Meeting ID: 910 8369 4183
Find your local number: https://cern.zoom.us/u/artUfAdNB
Join by SIP
91083694183@188.184.89.188
91083694183@188.185.118.153
Join by H.323
188.184.89.188
188.185.118.153
Meeting ID: 910 8369 4183
Passcode: 12345678
BiLD (Bi-weekly DIRAC Development meeting) – 18/08/2022
On Zoom: Federico, Christophe, Christopher, Cedric, Simon F, Simon M, Xiaomei, Janusz, Alexey, Ueda, Igor
Apologies: Andrei
Follow-up from previous meetings
- Last BiLD on June 30th, since then 2 hackathons.
- hackathon of July 7th, on 8.0.0a26:
- Missing dependecies on DIRACOS: https://trello.com/c/Sdu4nvTF/29-install-a-client-on-cc7-or-somewhere-else
- DataOperationSender bugs: https://trello.com/c/oWUqyn2G/18-submit-user-jobs-using-previous-dirac-version
- Not everything tested
- hackathon of August 4th, on 8.0.0a27:
- Still a few issues with Monitoring (ES) that were solved with PRs
DIRAC communities roundtable
GridPP:
SimonF+Janusz
- Using v7r3 and py3.
- Now using CloudComputingElement in production. Using Singularity too and PoolCE together (with patched version).
LHCb:
Federico+Christophe+Christopher+Alexandre+SimonM+Alexey
- Running latest v7.3.27
- MySQL DBs: Moved from BLOBs to TEXTs everywhere (a downtime was necessary)
- one issue found, PR created and merged
- Moved all ElasticSearch to OpenSearch (no issues)
- Have yet to run an LHCbDIRAC hackathon using DIRAC 8.0 as base
- from previous meeting Andrei to LHCb
- Further details about the optimizations performed would be interesting.
Federico details of the update could be found in https://codimd.web.cern.ch/5-ImZHjFTi6ko_a-5-5BFw#
- How many concurrent MySQL connections do you support?
- André 1000 should be the max
mysql> show session variables like '%connections%';
Variable_name Value
max_connections 1000
max_user_connections 1000
ILC/CLIC/FCC/Calice:
André
- NTR
EGI:
no one
- NTR
Belle2
Ueda
- Running on v7r2 since end of July (still py2)
- from previous meeting Anil Presented ideas on using the Rucio data popularity features in Belle II. A link to the presentation has been added.
BES, Juno
Xiaomei
- Just migrated to 7.3.20 (py3). Mostly fine, still some issues with using CloudComputingElement, still few issues – lots of progress thanks to SimonF.
Nica
Igor
- Everything is fine. Started migrated to v7.3.27. Added small fixes to documentation.
- Moving servers to bare metal
Topics from GitHub/Discussions or Google forum
only un-answered topics below:
- Is TimeOut handled in getFile API?
- Federico I don’t think it is something we want (change my mind!)
- Christophe there’s actually a timeout, I will answer
- Input file streaming
- Needs answer
- Dirac API supporting JDL submission
- Idea is approved, draft PR for 8.1 created (might be re-done)
- Error running singularity containers
- unanswered (done during the meeting)
- Expose rucio API via DIRAC (DMS/Client/RucioClient.py)
- unanswered
- Delays in pilot/JobAgent cycles
- “on hold”
- Access to pilot output and jdl files of other users
- from previous meeting Andrei No clear answer yet. Right now technically it is not possible to limit user to be able to see only their pilot outputs and JDLs. Pilot outputs can be fully protected from access by “Normal” users without special rights. Other job artefacts like JDLs can be accessed by users in a restricted way but this needs development.
- Submitting DIRAC jobs via HTTP
- “on hold”
- from previous meeting Andrei The proxies created with voms-proxy-init (outside DIRAC) can be still usable as DIRAC default group will be assumed on the SIRAC service side. This will be suitable for most of the users already. When we will move to tokens, the user metadata (claims, scopes) in the tokens will be mapped onto DIRAC groups even if tokens will be generated bypassing DIRAC as an Authorization Server.
- Authenticating to Tornado as a TrustedHost
- WIP
DIRAC releases
- v7r2:
- v7r2p48
- FIX: (#6223) Increase VOMS command timeout
- FIX: (#6265) Fix FC change{group,owner,mode} for multiple LFNs
- FIX: (#6230) remove REST information from the ARC6 pilot job references to bind them with jobs
- rel-v7r2 HEAD:
- uses
suds(only affects py3 clients)
- uses
- v7r2p48
- v7r3
- v7.3.26
- FIX: (#6274) Conditional FC ProxyPlugin checks the ThreadConfig
- v7.3.27
- CHANGE: (#6284) Make GOCDB2CSAgent functionMap less private
- Fix for ReqDB and BLOB (see above)
- rel-v7r3 HEAD:
- Few fixes merged (mostly Computing Resources), tag needed
- v7.3.26
- integration
- v8.0.0a27 tagged few weeks ago, since then many new fixes.
DIRAC projects
DIRAC:
Issues by milestone:
Other issues:
- Making use of Python typing
- “awesome”
PRs discussed:
- [8.0] Fix using HTTPS with Python 3.10+ and suppress InsecureRequestWarning
- Specific comment worth discussion: https://github.com/DIRACGrid/DIRAC/pull/6299#discussion_r946875312
- Christophe the transition could be made shorter by changing the required lifetime (with a hotfix)
- We agreed on proposing a “wiki” solution that would require some manual intervention.
- Specific comment worth discussion: https://github.com/DIRACGrid/DIRAC/pull/6299#discussion_r946875312
- [8.0] Ensure DIRAC isn’t configured before running unit tests
- Looks OK
- [8.0] Add arguments to DIRAC.initialize
- Needs review
- Draft: [integration] Remote Pilot Logger to Tornado
- For using a shifter proxy, a solution similar to what is used for the TransformationSystem can be used.
- In general: several PRs already waiting for v8.1, or better that there’s a
rel-v8r0branch so thatintegrationwill be used for v8.1 pre-releases
WebApp:
- Just 1 PR open, Christopher will fix it.
Pilot:
- PR [devel] feat: implement rolling pilot log output finally ready?
- seems so
- feat: Enable remote pilot logging using HTTP interface to Tornado
- from previous meeting Q: should we change also the DIRAC logger? in order to use the ISO 8601 format and microseconds. If this is done, then all log messages will have the same format
- an issue should be created with this
- file backend for now, and it will be enough for running the tests. Different backends (e.g. ES) can be added later
- from previous meeting Q: should we change also the DIRAC logger? in order to use the ISO 8601 format and microseconds. If this is done, then all log messages will have the same format
DIRACOS2:
- python 3.10 tests starting:
Documentation:
- from previous meeting diracgrid.org still forwards to http://mardirac.in2p3.fr
OAuth2:
- from previous meeting Any news from issue WLCG and Token transition – reminders and requirements ?
- timeline doc: https://docs.google.com/document/d/1Fv9LPZTKgg1Tn4pv06fvpj8EbakBEGw8Gu4VReq8OZ4/edit#
- Ueda X509 access to HTCondor will be kept (GSI support will be dropped). What does that mean? https://htcondor.readthedocs.io/en/v8_8/admin-manual/security.html#ssl-authentication
- Christophe are RFC proxies still supported? we only use those in DIRAC
tornado/HTTPs
- One release of our tornado fork for supporting python 3.10
management
- Added a few recipes (PRs already merged)
diraccfg
- Christopher Do we want to make a release that drops support for py2?
- yes
COMDIRAC
- Merge in DIRAC? https://github.com/DIRACGrid/DIRAC/discussions/6157
- from previous meeting Andrei This is a good idea to merge COMDIRAC but there are quite a number of subtle technical implications. Suggestion is to have a discussion among interested developers (Andrei, Andrii, Daniela, Simon) and come up with a proposal to be discussed at some future BiLD.
DB12
- NTR
Rucio
- [v7r3] Rucio integration is broken with Python 2
- Closed, only py3 will work
Release planning, tests and certification
8.0 release
- Create 8.0 release? LHCbDIRAC hackathon tomorrow, then we decide
- Christpher remove the various
sixfrom everywhere?- yes
Certification machines
- lbcertifdirac70 machine:
- NTR
- lbcertifdiracoauth machine:
- NTR
Next hackathon(s)
- from previous meeting Computing Element of each supported type should be configured (HTCondorCE, 2 ARCS (Alexandre), 2 Clouds (Daniela), SSH). Consider OAuth tests as well - what is needed as far as certification servers configuration is concerned.
AOB
Next hackathon on September 1st?
Next BiLD on September 15th?
from previous meeting An ARC/HTCondor CE Hackathon will happen on September 15th-16th in Amsterdam: https://indico.nikhef.nl/event/3612/. It would be good if one of us would participate.
LHCbDIRAC
- LHCbDIRAC Bookkeeping:
- One MR left to be merged
- LHCbDIRAC hackathon based on 8.0
- tomorrow morning
- v10r4: deploy board in https://trello.com/b/kzUKdMts/deploy-v10r3