WLCG DOMA BDT Meeting

Wednesday 2 Aug 2023, 16:30 → 17:30 Europe/Zurich

Brian Paul Bockelman (University of Wisconsin Madison (US)), Maria Arsuaga Rios (CERN), Petr Vokac (Czech Technical University in Prague (CZ))

Topic: WLCG DOMA BDT Meeting (twiki)

1 News

2 Tape REST access

Speaker: Mihai PATRASCOIU (CERN)

• dCache store all TAPE REST requests in the database
  • details about staging for each file in the individual request
  • TAPE REST release operation doesn't clean transfer details from database
  • TAPE REST allows to delete request which cleans database
  • FTS transfer from tape
    • call release operation after transferring file
      • since 3.12.8 release (FTS-1913)
    • never delete request
      • no concept of "finalizing" submitted transfers
      • it is up to TAPE REST implementation in the storage to cleanup old requests
        • robust TAPE REST implementations can't expect client always cleanup request
        • even if some client call request delete there might be still situations when cleanup is not done (e.g. client prematurely terminated)
  • current dCache implementation never cleanup requests without explicit delete request call
    • TAPE REST database just grows
      • since April ATLAS staged just ~ 2M files from FZK
      • database size doesn't cause troubles immediately, because performance with 100M records is still fine
      • bigger database -> slower responses to TAPE REST operations
        • with extreme DB size this could potentiality lead to timeouts
    • dCache developers will discuss improvements
      • some kind of automatic cleanup of the old requests

3 Transfers with tokens

DC24 transfers with tokens proposal and discussion

https://cernbox.cern.ch/s/CuCi1LUxgZ123pp

CERNBox folder for final documents

https://cernbox.cern.ch/files/link/public/aClTXJenZxpF5qw

Speakers: Petr Vokac (Czech Technical University in Prague (CZ)), Francesco Giacomini (INFN CNAF)

dCache behavior with tokens without storage.* scope

• ATLAS & CMS would like to use exclusively tokens
  • this seems to be more tricky than expected especially when combined with X.509 access that we still need during transition to tokens
• Long discussion about union of group + scope based authorization defined in WLCG JWT profile
  • we still did not update profile according discussion that was done in the past (email thread)
  • it seem current dCache implementation can be indeed configured to satisfy ATLAS & CMS expectation
    • prefer storage.* scope
    • reject access for tokens without storage.* scope
    • this configuration is technically not compatible with WLCG JWT profile v1.0
  • WLCG JWT profile needs to be updated / improved
• Fermilab is going to provide configuration example for dCache
  • it is possible to distinguish clients that comes with tokens including storage.* scope
  • gplazma.oidc.provider -authz-id configuration option is used only when storage.* scope is present in token

DC24 Transfer with tokens proposal

• as presented e.g. during last DOMA General it is necessary to write proposal what we would like to test during DC24
  • these proposals must be finalized before next DOMA General which will be held on August 30
• support for tokens not yet available (even implemented) for TAPE REST
  • tapes are out of the scope of this proposal
  • generally, tapes are not part of DC24
    • sites/T1 can still make their own proposal if they would like to do test with tapes during DC24 (e.g. some sites moves to dCache+CTA)
• started to work on transfers with tokens document draft

4 Packet marking

Speakers: Marian Babik (CERN), Shawn Mc Kee (University of Michigan (US))

5 WebDAV Error Message Improvement Project & unified error message format

Discuss with experts improvements in the error messages produced by failed transfers.
https://twiki.cern.ch/twiki/bin/view/LCG/WebdavErrorImprovement

Speaker: Stephan Lammel (Fermi National Accelerator Lab. (US))

6 AOB