WLCG DOMA BDT Meeting

Wednesday 1 Mar 2023, 16:30 → 17:30 Europe/Zurich

Brian Paul Bockelman (University of Wisconsin Madison (US)), Maria Arsuaga Rios (CERN), Petr Vokac (Czech Technical University in Prague (CZ))

Topic: WLCG DOMA BDT Meeting (twiki)

16:30 → 16:35 News

This meeting will be mostly focused on TAPE REST deployment status and plans

NOTES: possible topics for next meeting(s)

• archival metadata - standard for passing metadata to the tape storage (HTTP headers?) - probably too early for status update of this topic
• progress with recommended storage configuration for X.509 + tokens
  • ideas how to use tokens for user / group storage areas
  • issuer capability basepath

 

16:35 → 17:00 Tape REST access

Speaker: Mihai PATRASCOIU (CERN)

CERN CTA TAPE REST deployment plans

• start with LHCb
  • already using HTTP transfer + xroot check on tape
• move other experiments to TAPE REST next week
  • make CTA ready for TAPE REST transfers
    • Julien is in touch with ATLAS DDM Ops team
    • it is up to experiment to decide when they move to TAPE REST
    • ATLAS plans to move soon
  • glitch in check-on-tape found in Rucio/FTS (needs to be resolved for CMS)
• DAQ stays with xroot

Tape Archive Metadata (tape collocation and scheduling hints)

• Tape REST API & Tape Archive Metadata to be managed in Github repo. 
  • This will host documents (e.g.: PDF), documentation with important links, contact address and issues
  • Issues will be used as discussion and dissemination place
  • Best candidate: https://github.com/wlcg-storage
• Tape REST API & Tape Archive Metadata will be documented in different PDFs
  • A mention/reference from one PDF to the other will be added
• Discussed within group of FTS & CTA & dCache developers
  • work in progress - trying to understand experiment requirements and use-cases
    • define dataset collocation
    • needs to demonstrate that T1 gets sufficient metadata to profit from this project (FZK tests)
  • StoRM - currently no need for this interface
    • would like to see what is in standard
    • github page will be used to come with standard
  • in future there will be plugin interface for Rucio to provide these metadata

Tape transfers with token

• storage.stage scope not yet implemented
  • definition: Read the data, potentially causing data to be staged from a nearline resource to an online resource. This is a superset of storage.read.
  • needs to be discussed in more details
  •  implement as storage.read with permission to "bringonline" / use storage TAPE REST API

17:00 → 17:05 Transfers with tokens

Speaker: Francesco Giacomini (INFN CNAF)

ATLAS scope policies for transfer tests

• created new IAM optional groups
  • /atlas/policy - not used, just mandatory parent group
  • /atlas/policy/storage-test - for ATLAS SAM probes and SE tests
  • /atlas/policy/compute-test - for ATLAS SAM probes and SE tests
  • /atlas/policy/compute-pilot - for normal grid job submission (Harvester, aCT)
•  new groups to avoid too generic "production" group that's now used everywhere
• "internal" to the IAM (optional groups) that'll be used only for scope/token exchange policies
• storage policies
  • everything DENY by default
  • storage-test
    • allowed to get storage.read, storage.create, storage.modify
    • only for paths /atlasdatadisk/SAM/ and /atlasscratchdisk/SAM/
• ask P.V. (or VO-Admin) if you would like to get these test storage tokens
  • read/write/delete in test area of any storage configured to accept ATLAS tokens

CMS SAM tests for transfers with tokens

• CMS SAM token tests for WebDAV and XRootD endpoints
  • WebDAV: se_webdav.py (documentation)
  • xrootd: se_xrootd.py (documentation)
• recent XRootD client (5.5.2 or 5.5.3) required to avoid SEGV

17:05 → 17:15 Packet marking

Speakers: Marian Babik (CERN), Shawn Mc Kee (University of Michigan (US))

17:15 → 17:25 WebDAV Error Message Improvement Project

Discuss with experts improvements in the error messages produced by failed transfers.

Speaker: Stephan Lammel (Fermi National Accelerator Lab. (US))

https://twiki.cern.ch/twiki/bin/view/LCG/WebdavErrorImprovement

17:25 → 17:30 AOB