WLCG DOMA BDT Meeting

Wednesday 6 Sept 2023, 16:30 → 17:30 Europe/Zurich

Brian Paul Bockelman (University of Wisconsin Madison (US)), Maria Arsuaga Rios (CERN), Petr Vokac (Czech Technical University in Prague (CZ))

Topic: WLCG DOMA BDT Meeting (twiki)

16:30 → 16:35 News

16:35 → 16:45 Tape REST access

Speaker: Mihai PATRASCOIU (CERN)

16:45 → 17:00 Transfers with tokens

DC24 transfers with tokens proposal and discussion

https://cernbox.cern.ch/s/CuCi1LUxgZ123pp

CERNBox folder for final documents

https://cernbox.cern.ch/files/link/public/aClTXJenZxpF5qw

Speakers: Petr Vokac (Czech Technical University in Prague (CZ)), Francesco Giacomini (INFN CNAF)

DC24 Transfers with token plan

• DC24 document folder with basic proposal structure described in the README.TXT document (example on slide 8).
• Proposal for transfers with tokens during DC24
  • followup mail discussion about milestones for DC24 workshop

dCache improved transfer monitoring support

• fireflies preliminary support tested at AGLT1
  • issues found and it might still take quite some time to resolve them
• WLCG monitoring
  • rely on existing data about transfers provided by dCache
  • currently available data doesn't provide source IP address
  • future dCache version will provide also source IP address for (WLCG) monitoring purposes
    • first test at limited number of sites close to developers
    • not yet ready for wider deployment

dCache TAPE REST

• configuration for DATA vs. TAPE access with WLCG JWT access tokens with capabilities, we need
  • our namespace organization at T1 sites is usually
    • /prefix/DATA/vo/datadisk should be accessible with storage.{read,create.modify}:/datadisk
    • /prefix/DATA/vo/scratchdisk should be accessible with storage.{read,create.modify}:/scratchdisk
    • /prefix/TAPE/vo/datatape should be accessible with storage.{read,create.modify}:/datatape
    • /prefix/TAPE/vo/mctape should be accessible with storage.{read,create.modify}:/mctape
  • tricky to do something reasonable for T1 sites with following namespace organization
    • /prefix/vo/datadisk
    • /prefix/vo/scratchdisk
    • /prefix/vo/TAPE/datatape
    • /prefix/vo/TAPE/mctape
    • ask these T1 to change namespace e.g. to the layout mentioned above(?)
      • it would be necessary to keep in the mind that IAM should never provide token with /TAPE
        • this token could be used to access tape using disk doors
        • namespace organization at each site would have to be considered while configuring IAM
      • otherwise it would be necessary to be very careful while defining IAM scope policies for storage.* scopes
        • understand all differences in the namespace layout while defining policies in IAM - tricky
  • during last BDT while discussing storage.*:$PATH we came to conclusion that preferably LFN should (must) match $PATH (no additional site specific prefix for WLCG JWT token issuer)
    • might be tricky for some implementation
      • PIC namespace layout
        • disks directly at top level, e.g. davs://webdav-at1.pic.es:8446/atlasdatadisk
        • tape in the subdirectory, e.g. davs://webdav-at1.pic.es:8446/tape/atlasdatatape
      • FZK namespace layout
        • disks in the subdirectory, e.g. davs://atlaswebdav-kit.gridka.de:2880/pnfs/gridka.de/atlas/disk-only/atlasdatadisk
        • tape in the parent directory, e.g. davs://atlaswebdav-kit.gridka.de:2880/pnfs/gridka.de/atlas/atlasdatatape
    • is it possible with dCache to have disks&tapes at one directory level, e.g. drop "disk-only" from file path
      • disk: davs://atlaswebdav-kit.gridka.de:2880/pnfs/gridka.de/atlas/atlasdatadisk
      • tape: davs://atlaswebdav-kit.gridka.de:2880/pnfs/gridka.de/atlas/atlasdatatape
      • yes, this is technically possible
        • still not easy operation, because this might require site downtime
        • may be symlinks could allow smoother transition to different paths
    • we don't want to have "unnecessary" prefix visible in the URL (LFN)
      • hide prefix by configuring dcache.root (or webdav.root + xrootd.root) or omnisession root:/ for multi-VO dCache instance, e.g. for FZK use
        • davs://atlaswebdav-kit.gridka.de:2880/atlasdatadisk
        • davs://atlaswebdav-kit.gridka.de:2880/atlasdatatape
      • intuitive way to map storage.*:$PATH to the LFN
        • access to LFN /atlasdatadisk require simple storage.*:/atlasdatadisk/ scope in the access token
        • original /pnfs/gridka.de/atlas/atlasdatadisk would require either
          • storage.*:/pnfs/gridka.de/atlas/atlasdatadisk
            • really problematic for multi-VO with current WLCG JWT profile implementations
          • storage.*:/atalsdatadisk
            • site must provide what was configured as VO issuer prefix
            • current Rucio design (implementation plans) doesn't cover this case
    • e.g. for ATLAS we would like to have something like

	ATLASDATADISK	ATLASSCRATCHDISK	ATLASDATATAPE	ATLASMCTAPE
CERN	davs://eosatlas.cern.ch:443/atlasdatadisk	davs://eosatlas.cern.ch:443/atlasscratchdisk	davs://eosctaatlas.cern.ch:8444/atlasdatatape	davs://eosctaatlas.cern.ch:8444/atlasmctape
INFN-T1	davs://xfer.cr.cnaf.infn.it:8443/atlasdatadisk	davs://xfer.cr.cnaf.infn.it:8443/atlasscratchdisk	???	???
pic	davs://webdav-at1.pic.es:8446/atlasdatadisk	davs://webdav-at1.pic.es:8446/atlasscratchdisk	davs://webdav-at1.pic.es:8446/atlasdatatape	davs://webdav-at1.pic.es:8446/atlasmctape
FZK	davs://atlaswebdav-kit.gridka.de:2880/atlasdatadisk	davs://atlaswebdav-kit.gridka.de:2880/atlasscratchdisk	davs://atlaswebdav-kit.gridka.de:2880/atlasdatatape	davs://atlaswebdav-kit.gridka.de:2880/atlasmctape
PRAGUELCG2	davs://se1.farm.particle.cz/atlasdatadisk	davs://se1.farm.particle.cz/atlasscratchdisk	N/A	N/A

•  
  •  
    • right dCache configuration may be more complicated for CMS, because they have more complex namespace organization for their RSEs
      • may require different issuer prefix for different data areas
      • this is quite tricky / ugly to configure
        • multiple doors with different gPlazma configuration
        • complex and nightmare to maintain for site administrators

17:00 → 17:10 Packet marking

Speakers: Marian Babik (CERN), Shawn Mc Kee (University of Michigan (US))

Packet marking DC24 proposals:
https://cernbox.cern.ch/remote.php/dav/public-files/aClTXJenZxpF5qw/Packet_Flow_Marking.txt
https://cernbox.cern.ch/remote.php/dav/public-files/aClTXJenZxpF5qw/Packet_Flow_Marking_accounting.txt

17:10 → 17:25 WebDAV Error Message Improvement Project & unified error message format

Discuss with experts improvements in the error messages produced by failed transfers.
https://twiki.cern.ch/twiki/bin/view/LCG/WebdavErrorImprovement

Speaker: Stephan Lammel (Fermi National Accelerator Lab. (US))

17:25 → 17:30 AOB