Fallout from ATLAS Data Carousel

• GC fix (Steve):
  • FST aka "slow" GC fix is progressing. The "slow" GC removes files exceeding a configurable age without ordering. (The MGM "fast" GC is LRU and the migration GC evicts immediately after migration)
  • Steve has discussed with Elvin/Luca how to identify which files that have been completely read (ie. openened for reading, then closed). This would require substantial work (needs to be done in MGM but cleanup be issued on FST)
• Retrieve backstop: 
  • Proposal was discussed; Eric created a gitlab ticket with full details (#533, link)
• FTS cache/buffer removal:
  • requires an XROOT "un"-prepare method to do buffer cleanup (equivalent to "eos stagerrm").
  • Once "un-prepare" is available, FTS development latency is estimated between 1-2 weeks.
  • In principle, FTS would send this to all storage back-ends on the assumption this is ignored by non-CTA storage systems - this needs careful examination by FTS experts.
    • Note that providing storage specific back-ends is becoming increasingly relevant for FTS, as VO's such as ATLAS would like to pass metadata (storage class information, retrieve activities, collocation hints) via FTS to the back-ends.
  • How does FTS process file requests? While it can receive 100K transfer requests at once, it will process them in order (chunks of 5000 files). What happens with files that are pending due to e.g. a problematic tape or library? A timeout (900s) will kick in and files will be retried at a later stage.
• Others:
  • FTS production instance was used for recalls (instead of the -dev one) which was causing #streams to fall down after failures. Julien will check with FTS team to ensure the right FTS version is used for CTA activities for archivals and retrieves.

Action list: