Monitoring Updates (markdown)

• Limitations at measuring accurate throughput from the FTS aggregated data (ref.)

• Lack of metadata fields in the FTS aggregated data, this is needed to separate testing traffic from normal traffic (ref.) (this metadata exists in the aggregated data, thanks to Nick Smith for pointing it out)

• CMS Rucio data → retention policy of 1 month (hosted in the short term Elasticsearch cluster)

  • In contrast, ATLAS Rucio data → retention policy of 1+ year (hosted in InfluxDB)

  • Longer retention policy for the CMS case might be needed if we need a long-term monitoring → alternatively just use the FTS aggregated data and do not bother with Rucio data sources