WLCG DOMA BDT Meeting

Wednesday 22 Jun 2022, 16:30 → 17:30 Europe/Zurich

Brian Paul Bockelman (University of Wisconsin Madison (US)), Maria Arsuaga Rios (CERN), Petr Vokac (Czech Technical University in Prague (CZ))

Topic: WLCG DOMA BDT Meeting

Meeting notes written to individual topics.

1 Hot topics

2 Tape REST access

Speaker: Cedric Caffy (CERN)

Tape REST API status at CERN.pdf

RAL Antares (CTA) plans with TAPE REST API

• LHCb already accessing RAL Antares with WebDAV protocol
  • this protocol currently doesn't provide features for full tape system management
  • as a workaround xroot is currently used to fill this gap
  • sub-optimal multi-hop data transfers through Echo
    • sufficient for expected low 2022 data rates
    • RAL would like to transfer data directly for 2023 full physics run
• ATLAS & CMS currently use Antares with xroot protocol
  • practical point of view - use common protocol (HTTP-TPC) everywhere
  • these experiments could also benefit from early TAPE REST adoption at RAL

TAPE REST API timeline

• CERN Plan: Presentation for CTA+FTS in the attached slides
• RAL plan
  • have the production endpoint with TAPE REST API by February 2023
  • everything ready for 2023 Run (~ April)
• Both plans seems to have compatible timeline and looks realistic
• ATLAS use additional transfer metadata
  • must be passed from Rucio via FTS to TAPE REST API
  • some development necessary in Rucio for ATLAS
  • not used by other experiments
• Rough ATLAS plan for replacing SRM with TAPE REST
  • winter shutdown 2023/24 seems to be good opportunity to ask T1 for dCache upgrade => TAPE REST support
  • it is technically possible to start with early / active adopters of TAPE REST sooner (e.g. with RAL)
    • sites can be moved one by one once they become ready

Testbeds

• FTS will provide the tests to validate the REST API for the different endpoints.
• We (as BDT team) need to encourage the endpoints to join and complete the matrix.
• we would like to have
  • CERN devel CTA - already deployed
  • FNAL dCache with their implementation
  • RAL will also provide testbed because they are planning to be early adopters
• added TAPE REST API twiki page to document details

(Experiment) validation

• FTS is the service implementing the tests for validation.
• BDT will follow up the experiment validation.

 

• lightweigh - no changes in file transfers, just different call to bring file online
• REST API functionality tested by developers
• FTS functionality / Dirac direct interaction with TAPE endpoints validated together with experiments

3 Token Authorization testbed

Speaker: Francesco Giacomini (INFN CNAF)

JWT Compliance testbed

• added prague dCache 7.2.x and XRootD 5.4.x version
  • WLCG AuthN/Z documentation
  • dCache documentation issue#6607
  • OSG XRootD documentation
• plan to improve existing documentation for WLCG JWT tokens
  • allow more sites to start with WLCG JWT tokens
• describe minimal versions usable for HTTP / xroot transfers with tokens
  • XRootD - correct storage.create:/ support should come in 5.5
  • dCache
    • 7.2 provides separate OIDC and SCITOKEN plugin
      • WLCG JWT tokens supported by SCITOKEN plugin
      • sufficient support for our tokens
    • 8.2 provides more generic OIDC that can be used instead of SCITOKEN plugin
      • SCITOKEN plugin will be still available / supported for next ~ 2-3 years
      • very small changes required while moving from SCITOKEN plugin to OIDC
    • 8.2 xroot comes with Multiple authentication protocol chaining
      • looks like a useful feature for X.509 to tokens transition with xroot protocol
• we should start to advertise well in advance storage updates required for tokens
  • during data taking preferred windows for updates are winter shutdowns
  • dCache - sites could still use 7.2 with just configuration update
    • we should provide carrot
      • 8.2 can use improved OIDC plugin => different future proof configuration
      • 8.2 provides better support for xroot with tokens
    • we have to come with recommended OIDC identity mapping
      • compatible with concurrent X.509 usage(?)

4 Packet marking

Speakers: Marian Babik (CERN), Shawn Mc Kee (University of Michigan (US))

5 AOB

HTTP-TPC extensions (e.g. status code propagation, packet marking headers, ...)

• there are multiple reasonable requests to extend / improve HTTP-TPC standard
• WLCG is not the only user of HTTP-TPC
• form standardization committee that maintain this HTTP-TPC standard
  • transparent standardization process
  • clear way also for other communities to participate
• next BDT meeting - come with proposal how to organize such committee