Just a minor progress on SRM+HTTPs

• new test matrix with disk servers (dCache, DPM, EOS, Echo, StoRM, XRootD)
• fixed issue in Rucio and srm+https rucio#4650
• SH_* RSE with protocols: srm+https, https, davs
• 100MB file transfers every ~ 20 minutes (initial tests with 1MB, test bigger 5GB?)
• no issue related directly to srm+https
  • BNL srm+https destination caused by wrong RSE spacetoken configuration
    • which spacetoken can be used for dteam VO with srm://dcsrm.usatlas.bnl.gov:8443/srm/managerv2?SFN=/pnfs/usatlas.bnl.gov/users/hiroito/testtpc
  • TRIUMF most probably same firewall issue that we found also with prod transfers
  • RAL Echo still problematic
  • EOS firewall/configuration(?) on p06636710r84969.cern.ch
  • dCache prometheus - occaissional MAKE_PARENT failure (most probably related to deploying new release)

SRM+HTTP tape test - plan:

0) periodic tests only for SRM+HTTP and DISK (e.g. to understand occasional timeouts seen for transfer from DISK) - DONE
1) upload 100TB dataset (1GB, 10GB files) to several dCache/StoRM/EOS/DPM DISK storages (Rucio RSEs)
2) (Rucio) FTS transfer dataset with SRM+HTTP to StoRM TAPE endpoint (INFN-T1) and dCache TAPE endpoint (FNAL or BNL?)
3) ask tape admins to verify uploaded files are in right state, they reached tape and generally everything looks fine (or use SFO?)
4) ask tape admins to cleanup dataset files from disk buffer (to let bringonline in next step do real work)
5) (Rucio) FTS transfer with SRM+HTTP from TAPE storage to DISK endpoints dCache/StoRM/EOS/DPM
6) repeat only if we find an issue / from the point where we found issue

It'll be necessary to synchronize with tape storage admins - site ready for 2, local verification 3-4, site ready for 4 - I can imagine that all steps can take few weeks to complete, but first I would like to understand 0 (I'll replace tape with disk for [1]) and meanwhile I'll prepare test dataset.