Name: WLCG DOMA BDT Meeting
Start: 2023-08-16T16:30:00+02:00
End: 2023-08-16T17:30:00+02:00
Location: No location set

16:30 → 16:35

News 5m

16:35 → 16:45

Tape REST access 10m

Speaker: Mihai PATRASCOIU (CERN)

16:45 → 17:00

Transfers with tokens 15m

DC24 transfers with tokens proposal and discussion

https://cernbox.cern.ch/s/CuCi1LUxgZ123pp

CERNBox folder for final documents

https://cernbox.cern.ch/files/link/public/aClTXJenZxpF5qw

Speakers: Petr Vokac (Czech Technical University in Prague (CZ)), Francesco Giacomini (INFN CNAF)

Rucio & tokens
- Current plan is to use one powerful registered (IAM) client for all tokens obtained by Rucio
  - privileges: scim:read, storage.read:/atlasdatadisk, storage.create:/atlasdatadisk, storage.modify:/atlasdatadisk, storage.stage:/atlasdatadisk
  - shared token cache between Rucio services
  - it should not be difficult in future to separate privileges for individual Rucio services (if we find suitable reasons)
    - e.g. Reaper to use different IAM client allowed to get storage.modify tokens
    - all other Rucio services with access only to IAM client allowed to get only storage.{read,create}
- Guessing configured storage audience from URL
  - seems that each service comes with their own default values and clients have different expectations
    - WLCG JWT profile recommends scheme://hostname[:port]
    - OSG / HTCondor-CE by default use hostname:port
    - Rucio would like to use just hostname
  - ~~it may become necessary to configure storage with audiences like~~ https://wlcg.cern.ch/jwt/v1/any se1.farm.particle.cz se1.farm.particle.cz:443 https://se1.farm.particle.cz https://se1.farm.particle.cz:443 davs://se1.farm.particle.cz davs://se1.farm.particle.cz:443 se1.farm.particle.cz:1094 root://se1.farm.particle.cz roots://se1.farm.particle.cz xroot://se1.farm.particle.cz xroots://se1.farm.particle.cz root://se1.farm.particle.cz:1094 roots://se1.farm.particle.cz:1094 xroot://se1.farm.particle.cz:1094 xroots://se1.farm.particle.cz:1094 ~~to cover all expectations - that would be a bit verbose configuration~~
  - JWT optional aud can contain string or URI, but newer RFC8707 (standardize how to get aud claim in the token) mentions just absolute URI (grammar)
    - hopefully arbitrary string will be still compatible with RFC8707 issuer implementation
      - at least we can discuss with Indigo IAM devs this requirement when they implement issue#381
  - agreed that it'll be easier to use just hostname for audience on storage side
    - we'll ask sites to add hostname in a list of allowed audiences in the storage configuration while deploying token configuration for DC24
    - XRootD redirection on write - it is necessary to add "headnode" hostname to the list of allowed audiences also on disknodes
    - this could reduce storage audience configuration to just https://wlcg.cern.ch/jwt/v1/any se1.farm.particle.cz
- Storage path prefix for WLCG JWT token issuer configured by site administrator
  - also can't be currently automatically and reliably determined from file URL
  - Rucio would like to avoid yet another protocol configuration option - solutions:
    - reorganize namespace where VO always starts with top level "/"
      - this allows to keep same policy for $PATH in the storage.*:$PATH claims for whole infrastructure
      - IAM scope policies can be used to further restrict clients and e.g. never allow Rucio to get storage.*:/ token (only e.g. storage.*:/atlasdatadisk + storage.*:/atlasscratchdisk + ...)
        allows to define e.g. storage.*:/secret area with very limited access not even reachable by hacked Rucio
    - change storage implementation that issuer basepath configured on storage side prevents read/write operations
      - even storage.*:/ would not give access to whole storage but only to the issuer basepath
        this behavior would require changes in WLCG JWT profile
        also require (probably small) changes in storage implementations
        e.g. basepath /pnfs/example.com/atlas for ATLAS token issuer in dCache configuration would prevent modifications in /pnfs/example.com/cms area
      - Rucio anyway plans to use per RSE token audience
        different namespace structure is not technically issue for Rucio
        Rucio could easily ask for storage tokens with full path like, e.g. storage.*:/eos/atlas/atlasdatadisk
      - Storage areas not used by Rucio would need "clever" service with full knowledge of each storage namespace organization
        impossible to do with just IAM and scope policies
        development of another secure service that restrict which storage tokens can be obtained by user / client
    - CMS by design rely on first solution
    - ATLAS - P.V. would also prefer same approach as CMS
      - proposal to reorganize / unify ATLAS namespace dropped
  - discussion with examples - https://eosatlas.cern.ch:443/eos/atlas/atlasdatadisk
    - EOSATLAS configuration for ATLAS token issuer use prefix /eos/atlas
    - client is unable to guess what's configured on server side
      - lfn (authorization) vs. pfn issue
    - ideally superfluous prefix should be hidden from URL
      - with capabilities it is anyway impossible to do anything on /eos/atlas path
      - URL for EOSATLAS should be https://eosatlas.cern.ch:443/atlasdatadisk
      - INFN-T1 example davs://xfer.cr.cnaf.infn.it:8443/atlas/atlasdatadisk -> davs://xfer.cr.cnaf.infn.it:8443/atlasdatadisk
      - support for URL without issuer prefix in our grid storage implementations
        XRootD OK
        dCache OK
        StoRM should be also OK
      - removing prefix
        can be easy for files managed by Rucio (e.g. during downtime, for ATLAS it is necessary not to have also any running job)
        tricky if people access file directly / with some "filelist"
        it would be necessary to support both LFN for storage used directly by users (e.g. CERN)
    - reorganizing namespace significantly reduce number of sites usable for DC24 with tokens
FTS transfers with tokens
- describe in details token-exchange done requested by FTS (testbed with WLCG IAM?)
- start email thread with FTS & IAM team to clarify requirements / expectations
FTS & packet marking (see LHCONE R&D and RNT-WG)
- HTTP-TPC Update#2 - passing VO & activity to the storage
- This should now be ready for implementation
- We don't expect to use this functionality during DC24
  - packet marking will be done by storage dedicated to specific experiment
  - VO & activity will not come from these HTTP headers
    - this needs to be implemented in Rucio & FTS & storage
DC24 Transfers with token proposal (WIP)

17:00 → 17:10

Packet marking 10m

Speakers: Marian Babik (CERN), Shawn Mc Kee (University of Michigan (US))

Packet marking DC24 proposals:
https://cernbox.cern.ch/remote.php/dav/public-files/aClTXJenZxpF5qw/Packet_Flow_Marking.txt
https://cernbox.cern.ch/remote.php/dav/public-files/aClTXJenZxpF5qw/Packet_Flow_Marking_accounting.txt

RFC draft was presented at IETF in San Francisco in July by Dale Carder (ESnet)
- Draft: https://datatracker.ietf.org/doc/draft-cc-v6ops-wlcg-flow-label-marking/
- We received a number of positive comments from network experts (e.g. from Google)
- It was proposed that we look into possibilty to use IPv6 HBH option instead of flow label (this is being investigated)

Started preparations for SC23
- Dedicated meetings with R&Es on collectors, aggregators and dashboards
- Discussions and details in the last LHCONE R&D call (https://indico.cern.ch/event/1316019/)

17:10 → 17:25

WebDAV Error Message Improvement Project & unified error message format 15m

Discuss with experts improvements in the error messages produced by failed transfers.
https://twiki.cern.ch/twiki/bin/view/LCG/WebdavErrorImprovement

Speaker: Stephan Lammel (Fermi National Accelerator Lab. (US))

17:25 → 17:30

AOB 5m

Choose timezone

WLCG DOMA BDT Meeting