DC24 Transfers with token plan

• DC24 document folder with basic proposal structure described in the README.TXT document (example on slide 8).
• Proposal for transfers with tokens during DC24
  • followup mail discussion about milestones for DC24 workshop

dCache improved transfer monitoring support

• fireflies preliminary support tested at AGLT1
  • issues found and it might still take quite some time to resolve them
• WLCG monitoring
  • rely on existing data about transfers provided by dCache
  • currently available data doesn't provide source IP address
  • future dCache version will provide also source IP address for (WLCG) monitoring purposes
    • first test at limited number of sites close to developers
    • not yet ready for wider deployment

dCache TAPE REST

• configuration for DATA vs. TAPE access with WLCG JWT access tokens with capabilities, we need
  • our namespace organization at T1 sites is usually
    • /prefix/DATA/vo/datadisk should be accessible with storage.{read,create.modify}:/datadisk
    • /prefix/DATA/vo/scratchdisk should be accessible with storage.{read,create.modify}:/scratchdisk
    • /prefix/TAPE/vo/datatape should be accessible with storage.{read,create.modify}:/datatape
    • /prefix/TAPE/vo/mctape should be accessible with storage.{read,create.modify}:/mctape
  • tricky to do something reasonable for T1 sites with following namespace organization
    • /prefix/vo/datadisk
    • /prefix/vo/scratchdisk
    • /prefix/vo/TAPE/datatape
    • /prefix/vo/TAPE/mctape
    • ask these T1 to change namespace e.g. to the layout mentioned above(?)
      • it would be necessary to keep in the mind that IAM should never provide token with /TAPE
        • this token could be used to access tape using disk doors
        • namespace organization at each site would have to be considered while configuring IAM
      • otherwise it would be necessary to be very careful while defining IAM scope policies for storage.* scopes
        • understand all differences in the namespace layout while defining policies in IAM - tricky
  • during last BDT while discussing storage.*:$PATH we came to conclusion that preferably LFN should (must) match $PATH (no additional site specific prefix for WLCG JWT token issuer)
    • might be tricky for some implementation
      • PIC namespace layout
        • disks directly at top level, e.g. davs://webdav-at1.pic.es:8446/atlasdatadisk
        • tape in the subdirectory, e.g. davs://webdav-at1.pic.es:8446/tape/atlasdatatape
      • FZK namespace layout
        • disks in the subdirectory, e.g. davs://atlaswebdav-kit.gridka.de:2880/pnfs/gridka.de/atlas/disk-only/atlasdatadisk
        • tape in the parent directory, e.g. davs://atlaswebdav-kit.gridka.de:2880/pnfs/gridka.de/atlas/atlasdatatape
    • is it possible with dCache to have disks&tapes at one directory level, e.g. drop "disk-only" from file path
      • disk: davs://atlaswebdav-kit.gridka.de:2880/pnfs/gridka.de/atlas/atlasdatadisk
      • tape: davs://atlaswebdav-kit.gridka.de:2880/pnfs/gridka.de/atlas/atlasdatatape
      • yes, this is technically possible
        • still not easy operation, because this might require site downtime
        • may be symlinks could allow smoother transition to different paths
    • we don't want to have "unnecessary" prefix visible in the URL (LFN)
      • hide prefix by configuring dcache.root (or webdav.root + xrootd.root) or omnisession root:/ for multi-VO dCache instance, e.g. for FZK use
        • davs://atlaswebdav-kit.gridka.de:2880/atlasdatadisk
        • davs://atlaswebdav-kit.gridka.de:2880/atlasdatatape
      • intuitive way to map storage.*:$PATH to the LFN
        • access to LFN /atlasdatadisk require simple storage.*:/atlasdatadisk/ scope in the access token
        • original /pnfs/gridka.de/atlas/atlasdatadisk would require either
          • storage.*:/pnfs/gridka.de/atlas/atlasdatadisk
            • really problematic for multi-VO with current WLCG JWT profile implementations
          • storage.*:/atalsdatadisk
            • site must provide what was configured as VO issuer prefix
            • current Rucio design (implementation plans) doesn't cover this case
    • e.g. for ATLAS we would like to have something like

•  
  •  
    • right dCache configuration may be more complicated for CMS, because they have more complex namespace organization for their RSEs
      • may require different issuer prefix for different data areas
      • this is quite tricky / ugly to configure
        • multiple doors with different gPlazma configuration
        • complex and nightmare to maintain for site administrators