BiLD (Bi-weekly DIRAC Development meeting) – 03/11/2022
On Zoom: Federico, André, Andrei, Alexey, Alexandre, Christopher, Daniela, Hideki, Igor, Michel, Simon, Janusz
Apologies: Xiaomei
Follow-up from previous meetings
DIRAC communities roundtable
LHCb:
Federico+Alexandre+Christopher+Alexey
- Running in production v7.3.32
- PR #6477 that backported ElasticJobParametersDB from rel-v8r0 branch was necessary because the single index used in LHCb was too large
ILC/CLIC/FCC/Calice:
André
EGI:
Andrei
Belle2
Hideki+Ueda
- Production on v7r2. Ready to use ElasticSearch
NICA
Igor
https://github.com/DIRACGrid/DIRAC/issues/6474 (see below)
during transfer to xrootd DIRAC uses parameter xrd.gsiproxy=/tmp/x509. In our JINR installation these transfers are blocked. Using parameter xrd.wantprot=gsi,unix seems to work stable. Issue opened in xrootd repository.
GridPP:
Daniela+Simon+Janusz
- No changes in production: v7.3.26 on production server
- v8.0.2 in testing: deployed with a matching indigo iam server for token testing
- (incidentally DIRAC UI seems to work fine if installed on Rocky 9)
Topics from GitHub/Discussions or Google forum
only un-answered topics below:
DIRAC releases
- v7r3
- v7.3.32
- WorkloadManagementSystem
- NEW: (#6477) backported ElasticJobParametersDB from rel-v8r0 branch
- FIX: (#6469) reduce the PollingTime of JobAgent to 20s
- Core
- FIX: (#6468) quoting the DB passwords for special characters
- Resources
- NEW: Added AREXComputingElement
- v8r0
- v8.0.3
- Will be created when current PRs are approved+merged
- v8r1
- v8.1.0a3
- CHANGE: (#6451) remove support for myProxy
- CHANGE: (#6380) Remove VMDIRAC components
DIRAC projects
DIRAC:
Issues by milestone:
- v7r3:
- v8.0:
- 14 open issues, some of them will need to be re-assigned, and/or moved
- v8.1:
Other issues:
PRs discussed:
WebApp:
- 4 PRs open, mostlty waiting for reviews
Pilot:
DIRACOS2:
- 3 new releases in the past 2 weeks:
- 2.29
- Downgraded
arc library due to buggy arc release reported here - Update tests to test arc even more thoroughly
- 2.28
- Fix issue generating macOS installer
- Add missing arc dependency
- Update tests to test arc more thoroughly
- 2.27
- NEW: (#81) Include apptainer as a singularity alternative.
- from previous meeting Add htcondor-utils current version (9.11.2), fix diracosrc to use GSI and SCITOKENS auth methods. PR will be added for the latter.
- There is no version of HTCondor that supports both
- Christopher we can ask for re-packaging of old versions in Conda.
- from previous meeting Christopher if anyone is interested in making a CVMFS-based environment, let me know I will give instructions
Documentation:
OAuth2:
tornado/HTTPs
- Created 3 new tornado handlers in PR 6450, for FrameworkSystem
- from previous meeting v8.0.1 WebApp ssl issue
- PR with fix (draft) created in WebApp, Chistopher care to test?
- Simon I tested and looks OK
- Christopher I would still not be very confident about merging this PR
- Christophe …did not we say in the past that NGINX is becoming mandatory?
- Daniela if we can get away without it, it would be better
- There’s no log about this decision even though it was definitely discussed.
- Christophe
tornado is crap for download, nginx “needed” in this sense
- Andrei do we make nginx mandatory? In this case we should provide proper instructions. Then it should be run on every host (also for https services)
- 03/11 Daniela As a side note: GridPP would probably run this with apache as we use that elsewhere.
- Federico in practice what does that imply?
management
- from previous meeting 3 issues left, still valid
diraccfg
- from previous meeting Christopher Do we want to make a release that drops support for py2?
COMDIRAC
- Daniela Still working on merge (basically going through the commands one by one, fixing them up as we go along, hoping to evade the wrath of Federico™).
- For discussion:
dinput (command to retrieve input sandbox or jdl): Can we drop this ?- Andrei for the moment it can be dropped
DB12
Rucio
Release planning, tests and certification
8.1 integration tests:
Certification machines
- lbcertifdirac70 machine:
- Computing Element of each supported type should be configured (HTCondorCE, 2 ARCS (Alexandre (maybe use RALPP and Glasgow ? I can talk to them if there are issues --Daniela)), 1 Cloud (Daniela - this is already part of the tests), SSH). Consider OAuth tests as well - what is needed as far as certification servers configuration is concerned.
- lbcertifdiracoauth machine:
*
Next hackathon(s)
- Next week, on lbcertifidirac70 and with v8.1.a0X
AOB
- CHEP2023 seems to be happening for real this time. Abstract submission deadline November 17th (very soon!). Leaving it here, proposals welcome.
- prob extended to November 28th
Next hackathon: November 10th
Next BiLD: November 17th
LHCbDIRAC
- v10r4: deploy board in https://trello.com/b/kzUKdMts/deploy-v10r3
lbprod update: should we move ProxyDB to another instance (separate from the rest)?
- https://lhcb-auth.web.cern.ch/
- We need to have IaM as interface of VOMS (some code need to be developed here)
- For the actual operation we should see (up to Joel).
- Andrei this is not properly configured yet. Not usable for Pilot submission yet (compute scopes not added).
- Christophe is involved, probably should wait for him.
- LHCbDIRAC hackathon based on 8.0
- Next when? – next Tuesday at 14:00
- At some point (after it will be done on DIRAC certification) we need to setup a CE that accepts token and try out our IAM instance.
- ISIMA projects anyone?
There are minutes attached to this event.
Show them.
