BiLD – 31/10/2024

At CERN: Federico, André, Christopher, Alexandre, Ryunosuke, Cedric
On Zoom: Simon, Daniela, Hideki, Andrei, Vladimir, Xiaomei, Janusz
Apologies:

Follow-up from previous meetings

Last BiLD was October 3rd
Last DIRAC certification hackathon on October 10th
- Board: https://github.com/orgs/DIRACGrid/projects/15
- did not go very far, but enough to find out several issues
CHEP 2024 19th-25th Oct, Krakow
- Federico points I retained:
  - There is a WLCG Token Trust and Traceability WG that provides the same recommendations that we always heard for proxies ( Daniela: Well, it tried to solve the same problems…)
  - IAM talk. Daniela: Development still driven by WLCG, technical debt, MFA now implemented. It looks like the developers would like to have a proper multi-VO IAM as they clearly get pressured by their own immediate employer to do so :-)
  - Alice is doing interesting stuff with their full-node submission
  - Alice implemented cgroups for subdivision of WNs (we should do the same)
    - task for DIRAC: https://github.com/DIRACGrid/DIRAC/issues/7853
  - “The correlation between HS23 and DB12 (whole) is “almost” acceptable”
  - CMS is overloading
    - Daniela would like to note that LHCb should not be overloading at the Tier2s. If you look at the actual numbers, CMS efficiency even with overloading rarely tops 75% at the Tier2s, while e.g. here at Imperial LHCb hovers around 98%
      - Federico don’t worry, LHCb was not thinking about doing the same!
  - I found only one talk from DUNE
    - Yes, JustIN, by LHCb export A McNab. (Comment by Daniela)
  - Daniela Apart from Federico’s plenary DIRAC was also mentioned in Xiaomei’s JUNO plenary, and twice during parallel sessions (CTA & HERD)

DIRAC communities roundtable

LHCb:

Federico+Alexandre+Christopher+Vladimir

Stressing the Transformation System, applying patches
- having thousands of Transformations, some of them having 10M+ jobs
- TransformationAgents seem “fine”, WorkflowTaskAgent performances need to improve
Stressing also the WMS, some jobs running twice (same jobID picked up by 2 different pilots)
- Federico will re-check
Sending jobs to message broker stops the rest, if the broker is down issues arise. Timeout added, but single lock around logging is problematic
- patched in latest release
error message coming from M2Crypto, sending negative data
- also patched in latest release

ILC/Calice/FCC

André

EGI

Andrei

Setting up token based pilot submission with the production instance of Check-In (biomed)
Setting up the diracx test system with K3S in parallel with DIRAC 9.0. Struggling with setting up the basic setup (cert-manager, DEX, …)

Belle2

Hideki, Cedric

Completed migration to EL9, and to OpenSearch (still v1)
- will use apptainer

GridPP:

Daniela, Simon, Janusz

We would like to upgrade to the latest v8 version, but we have too many problems with unrenewed proxies. Not every VO went down the WLCG route of 7 day proxies, so we tend to see them more often: https://github.com/DIRACGrid/DIRAC/discussions/7842
- Andrei the current CEs are not renewing the pilot proxy, so at the moment the only way is to have long pilot proxies
  - Alexandre ARC is doing it (upgrading the delegation), I do not think HTCondorCE does it. Anyway this would not help with the bundled proxy that we are using now
  - proxy renewing itself? does not seem “correct”
  - Andrei proxy delegation thread in SiteDirector?
  - or maybe a Dirac-only internal token/proxy (without VOMS) only for renewing the user proxy
    - maybe re-using the same solution used for the CloudCE. Simon will have a look

Topics from GitHub/Discussions

only un-answered topics with discussion updates:

How to install TokenManager
- Andrei can backport PR [9.0] DISET version of the TokenManager service
Any update for https://github.com/DIRACGrid/DIRAC/discussions/7163 ?
- Can be closed Daniela

DIRAC releases

v8r0
- 8.0.56
  - truly “fixes”, of (really) old bugs
- 8.0.55
  - NEW: Use getTransformationFilesAsJsonString for faster getTransformationFiles

DIRAC projects

DIRAC:

Issues by milestone:

v8.0:
- job status transition in ReqClient vs JobStatus
  - Andrei will answer
- Removal of “Systems” part of CS
  - PRs for v8 and v9 merged
v9.0:
- NTR

Other issues:

from previous meeting Pilot status for pilots in PollTime related sleep cycle
- interesting discussion, feel free to contribute

PRs discussed:

https://github.com/DIRACGrid/DIRAC/pull/7848
- can it close the issue? – yes
- swept? – now yes
[sweep:integration] feat (TS): add getTransformationFilesAsJsonString
- needs update
- all other PRs targeting integration waiting for this to be merged

WebApp:

Sencha request (how it developed)
- Sencha made an official request “to EGI” asking for license explanation. After some exchange, Sencha got convinced that a paid license is not needed
Upgrade ExtJS to 6.6? (right now we are using 6.2)
- …maybe NOT! (and no volunteers!)

Pilot:

from previous meeting Janusz some doc to write

DIRACOS:

M2 support is there
from previous meeting Installation errors for diracos

Documentation:

from previous meeting Need to decide on strategy for DiracX documentation – André to take care?
- MkDocs ?

OAuth2:

management

from previous meeting Always upload releases to CVMFS
- still not working (did not work for for 8.0.53)
- Andrei created a new script, so PR needed
Daniela We just had a problem for a different VO (comet) where cvmfs would not distribute newly created directories, but happily update new content in old ones - not sure if this could be related ?
- not related

diraccfg

DB12

Igor made studies

Rucio

Tests

DiracX:

Issues

MySQL connection fails if inactive for a while
- Can it be closed? --> not yet
https://github.com/DIRACGrid/diracx/issues/30
- updates? --> not much

PRs discussed:

https://github.com/DIRACGrid/diracx/pull/297
- what to do?

DiracX-charts:

DiracX-web:

Alexandre
- Improvements to JobMonitoring app in a large-ish
- Added the user’s details at login
- Some tests for the extension needs to be added
asked Ryunosuke to be a reviewer for the DiracX-web PRs

Release planning, tests and certification

Certification machines
- No updates, using the old setup at CERN:
  - lbcertifdirac70 for DIRAC code
  - DBs from CERN
  - DiracX running on paas.cern.ch (OpenShift)
Next hackathon(s)
- next week

Next appointments

Meetings:
- hackathon: next week
- BiLD: in 2 weeks
WS/hackathons/conferences:
- DiracX hackathon: 14th-15th January 2025, CERN https://indico.cern.ch/event/1458873/
- Dirac&Rucio mini-workshop and hackathon: 16th-17th January 2025, CERN https://indico.cern.ch/event/1443765/
- Dirac Users’ Workshop: 17th-20th September 2025 - https://indico.cern.ch/e/duw11

AOB

Projects for ISIMA students
- Deadline: mid of November

LHCbDIRAC

BKK:
- on
  - Several queries taking way longer than what they should
- Chris “won’t survive another year” in the current conditions
- ConsistencyChecks performance improved
Ryun possibility of running on a subset of a BkQuery for prescaling
- “Merged”
Ryun’s DAG graphs: we can run “DAGs” in productions (sequentially)
- https://gitlab.cern.ch/-/snippets/3303
- CWL, one day
from previous meeting Some runs without luminosity. Total luminosity is there.
- Frèderic Hemmer to check it