WLCG DOMA BDT Meeting

Wednesday 6 Jul 2022, 16:30 → 17:30 Europe/Zurich

Brian Paul Bockelman (University of Wisconsin Madison (US)), Maria Arsuaga Rios (CERN), Petr Vokac (Czech Technical University in Prague (CZ))

Topic: WLCG DOMA BDT Meeting

16:30 → 16:35 News

BDT documentation

16:35 → 16:50 Transfers with tokens

Speaker: Francesco Giacomini (INFN CNAF)

TODO: move these informations in BDT twiki

Transfers with tokens & X.509

Do we expect to run in parallel transfers authorized with X.509 using different roles and tokens? What would be the correct identity mapping in this case and does current SE implementation provides sufficient flexibility?

For DC23 we'll need to use both X.509 and tokens at same time to be able to use production storages.

Is current token only testbed sufficient to cover concurrent X.509 + token usage? ... we should try to cover use-cases that we need by our production infrastructure not only for DC23, but also to cover future transition from tokens to X.509 which might not happen over night.

Plans not to shift DC23 too much, but realistically it may even be postponed after winter conferences ... nothing official yet, we should still focus to find potential implementation issues / missing features ASAP, because of long design-develop-test-deploy cycle.

Same (sub)namespace for all storages

Capability based security defined by storage.*:/path scope.

• prefix / base path can be different - configurable per issuer
• everything below must follow same structure
  • currently not true for all ATLAS RSEs, we need everywhere same structure
    • /arbitrary/prefix/atlasdatadisk
    • /arbitrary/prefix/atlasscratchdisk
    • /arbitrary/prefix/atlaslocalgroupdisk
    • ...
  • even different spacetokens for data vs. scratch ("analysis area") vs. localgroup (institutional resources) disk
  • symlinks could help with transition to the new structure
    • supported in dCache
    • Echo, EOS, XRootD - can use plugin for LFN mapping (already in use by CMS)
    • StoRM

File access permissions

Only one identity for clients using storage.*:/ scopes (capability), because we can't map identity according different path restrictions in the storage scope. Is it possible to come with configuration that allows X.509 and tokens live in one storage namespace without introducing security permission issues?

Different storage areas currently use different permissions (data vs. scratch vs. localgroup).

• dCache
  • does dCache consider ACLs for capability based access?
    • no, capability overwrite ACL checks
  • possible by configuring inherited ACLs
    • dCache can't map identity using (sub)path matching (like XRootD)
• XRootD
  • possible with one user and acc.authdb permission mapping
  • not clear for storage using xrootd-multiuser
    • could we rely on backend filesystem ACLs (posix, hdfs, ...)?
    • needs imput from developer / site operators
• StoRM
  • not clear to me if fine grained authorization provides sufficient features
  • together with defined default ACL entries everything should work for ATLAS
    • not sure if filesystem configuration is flexible enough for others
  • details to be discussed
• Echo
  • similar situation like in EOS case
    • we think current XrdSciTokens "name_mapfile" should be sufficient
  • details to be discussed
    • private email communication
    • we'll see on compliance testbed
• EOS
  • rely on grid-mapfile with very simple mapping
    • done on purpose - for full compatibility with CASTOR
    • technically EOS / XRootD could be configured to directly use VOMS extension
      • we can live with current model for next few years
      • it'll be replaced (no longer used) once we fully move to tokens
  • special mapping for a few (service) accounts
    • for compatible access with tokens it'll be sufficient to use XrdSciTokens "name_mapfile" with few lines for special accounts ("sub") or may be even better to map path to specific XRootD account, e.g.
      • "path": "/atlas/atlasscratchdisk" -> atlas001 (users)
      • "path": "/atlas/atlasdatadisk" -> atlas003 (production)
  • storage.*:/ scope capability doesn't always give client access
    • mapped user identity is validated later by XRootD
    • this is different behavior compared e.g. to the dCache
      • WLCG JWT compliant EOS behavior require precise identity mapping
  • technically EOS also supports ACLs
    • not used for LHC experiments
    • could be useful to solve complicated identity mapping scenarios
      • fortunately LHC experiments use very simple mapping and can live without ACLs

Provide to the developers how exactly we use different X.509 identities so they will be able to tell if/how storage can be configured for mixed usage with tokens.

As an output for this effort provide documentation with details how to configure sites with common SEs for concurrent X.509 and token usage.

ATLAS requirements

Discussed in a project-lcg-authz mailing list: "SE token deployment/development"

Recommended storage configuration

Does LHC experiments have their documentation how to correctly configure their storage? Please provide links here:

• ATLAS - not aware of any documentation with technical details except generic ATLAS Sites Setup and Configuration and ACL configuration for DDM - identity for everybody (analysis data), production data and multiple national identities (but storage usually supports just space for members of one national group)
• Alice - single user on storage side (already using capability security model)
• CMS - according DMWMPG Namespace and dCache configuration example everything is mapped to the few storage identities, user area usually share one specific identity but its up to the site to decide in this case
• LHCb - just production user with read/write everywhere and normal users with write privileges in a specific subdirectory

How do you plan to use tokens for storage access?

• ATLAS - capability only
• CMS - capability comes with Rucio
• Alice - already rely on capability
• LHCb - prefers capability but depends on Dirac development / decisions how to deal with tokens

Compliance testbed

• recent results (last results)
• need one more iteration for compatible prague-xrootd configuration
  • where is the patch for correct storage.create:/ functionality?
• come with dCache 7.2 prague-dcache configuration compatible with compliance testbed next time
  • issue with missing identity mapping in case of no storage.*:/ scope in the token
• configuration for remaining storage implementations(?)

16:50 → 17:05 Tape REST access

Speaker: Cedric Caffy (CERN)

Twiki page with basic informations about TAPE REST. Please add details about testbeds and try to aggregate there all information for this interface and how to use it.

Fixed REST endpoints for FNAL dCache setup

DESY may provide second test dCache endpoint.

FTS / gfal2 developers now have second implementation to test

• developing client for CTA (TAPE REST) revealed few unclear part of specification
• some additional dark corners expected in the other implementation
• don't want to claim development finished unless it supports multiple implementations

Token support in FTS for TAPE REST

• not on the schedule for September TAPE REST support in FTS (evicet, archive, ...)
• this will come later - right now focus on X.509 which we would like to use in production already during 2023

It was suggested we should come with automatic tests for TAPE REST API similar to WLCG JWT Compliance

• FTS developers happy to provide tests, but only for functionality they use
  • they don't use full TAPE REST API, because FTS keeps a lot of information internally (it doesn't have to ask for them via TAPE REST API)
• keep remaining TAPE REST functionality tests on TODO list for future

17:05 → 17:20 Packet marking

Speakers: Marian Babik (CERN), Shawn Mc Kee (University of Michigan (US))

Tests with packet marking using eBPF filters between AGLT2 and CERN(?) successful. Plugin for flowd in development. More details probably after summer.

17:20 → 17:30 AOB

WLCG IAM instance updated on Monday to the latest 1.8.0 release - please try / test it.