WLCG DOMA BDT Meeting

Wednesday 15 Mar 2023, 16:30 → 17:30 Europe/Zurich

Brian Paul Bockelman (University of Wisconsin Madison (US)), Maria Arsuaga Rios (CERN), Petr Vokac (Czech Technical University in Prague (CZ))

Topic: WLCG DOMA BDT Meeting (twiki)

16:30 → 16:35 News

16:35 → 16:45 Tape REST access

Speaker: Mihai PATRASCOIU (CERN)

Deployment status

• on Monday ATLAS CTA will be updated and TAPE REST enabled (OTG0076390)
• FZK discovered dCache issue which is now fixed - part of next 8.2.17 (GGUS:160869, dCache#13915)
• LHCb status & plans? CMS plans?

Support for tokens - storage.stage:/

• what happens if request contain some paths outside the storage.stage:$PATH capability?
  • should whole TAPE REST request fail or fail just for paths that are not covered by given storage.stage capabilities?
• what happens if request doesn't have storage.stage:$PATH capability at all, but (some) files are still in disk buffer?
  • should whole TAPE REST request fail or fail just for files that are not already in disk buffer?
• dCache provides stage protection where users (or some protocols) may be allowed to read data from disk buffer but they are not allowed to stage
•  

16:45 → 17:00 Transfers with tokens

Speaker: Francesco Giacomini (INFN CNAF)

Progress with recommended storage configuration for X.509 + tokens

• CMS started with SAM tests using tokens (found by reading dCache-support#10437)
  • yes, started to commission EOS with tokens
  • XRootD sites are next, followed by dCache and StoRM
  • issues with tokens stored in a file with newline character(s)
    • not stripped by our clients (gfal2, xrd, ...) and break HTTP request headers
    • this may be very common mistake
      • CMS prefers if clients gets more "clever" and always strip newline from passed token content
      • client could also just report reasonable error message
• ideas how to use tokens for user / group storage areas
  • scopes - default vs. restricted vs. normal
    • CMS still rely on restricted storage.*:/ scopes + scope policies
  • discussion about client_credentials
    • currently any user can configure this grant type for a client
    • this grant type should be configurable only by IAM Admin (similarly to the token exchange grant type)
      • it may be "dangerous" to let random user create registration with client_credentials grant type
      • limited protection in OAuth for this grant type
  • IAM user can register arbitrary undefined scopes iam/issue#546
    • allows application developers to test their code with new scopes
    • IAM should always use some prefix for these user defined scopes, e.g. test:my_scope
• storage issuer basepath set to "" makes easier to understand which URL path can be accessed with given storage claim path restriction
  • URL $PATH & storage.*:$PATH would be same
  • e.g. https://se1.farm.particle.cz/atlas/atlasdatadisk -> reorganize namespace -> https://se1.farm.particle.cz/atlasdatadisk
    • basepath /atlas is stored in SE configuration for ATLAS issuer - make it completely hidden from clients
    • otherwise "user" needs to know what's stripped from URL and where storage.*:$PATH applies
      • e.g. how can user know that storage.read:/atlasdatadisk applies directly in https://se1.farm.aprticle.cz or https://se1.farm.particle.cz/arbitrary/prefix/for/atlas/vo
    • simplify - everything starts with "/"
    • easily configurable with dCache rootdir for mapped issuer identity
      • already used by some sites - same URL points to the different part of namespace for different identities (VOs)
  • full path would be still required for open access in case we would like to publish some data anonymously
  • CMS namespace can't be reorganized this way for shared TAPE & DISK endpoint
    • OPTIONAL - start VO namespace directly in "/"
    • low priority
  • complexity with possible different URL vs. storage.* path is hidden by tools like Rucio
    • changing namespace to start in "/" for each VO most probably not generally worth of effort

17:00 → 17:10 Packet marking

Speakers: Marian Babik (CERN), Shawn Mc Kee (University of Michigan (US))

We are arranging a meeting with the dCache team to discuss packet/flow marking

Started testing new flowd package at AGLT2.   Will be in touch with BNL next

Austin (UM Student) is testing within Kubernetes on the FABRIC infrastructure

HEPiX and LHCOPN/LHCONE meetings coming up and we will be presenting and discussing there.

  -  SC23 plans in the works (include FTS/Rucio?)

 

17:10 → 17:25 WebDAV Error Message Improvement Project & unified error message format

Discuss with experts improvements in the error messages produced by failed transfers.
https://twiki.cern.ch/twiki/bin/view/LCG/WebdavErrorImprovement

Speaker: Stephan Lammel (Fermi National Accelerator Lab. (US))

17:25 → 17:30 AOB

HTTP-TPC transfers to S3 endpoints needs multipart support to be able to accept 5GB+ files

• implemented in Davix (DMC-1134, DMC-1135, source)
• HTTP-TPC push => PUT in the storage implementation
• not available in dCache, ??? (to be checked / discussed)
  • this needs to be tested and fixed
  • seems to work fine with EOS (XRootD)