Indico celebrates its 20th anniversary! Check our blog post for more information!

18–19 Mar 2024
CERN
Europe/Zurich timezone

Improving CTA performance by splitting user and repack scheduler backends

19 Mar 2024, 11:30
15m
31/3-004 - IT Amphitheatre (CERN)

31/3-004 - IT Amphitheatre

CERN

105
Show room on map
15 Minutes CTA Operations CTA Development and Roadmap

Speaker

Joao Afonso (CERN)

Description

Recent improvements to CTA repack added several new features and tools for operators. However, we were still faced with severe performance issues when repacking tapes on a very large scale. An investigation showed that this was mostly due to limitations on the CTA SchedulerDB backend, which did not scale well to performing repack on the latest generation of very high capacity tapes, which can store 50 TB of data and millions of files. As a mitigation, while a new SchedulerDB is still in development, we decided to split the "user" and "repack" scheduler backends. This will effectively prevent repack jobs from interfering with user archival jobs. In this talk, we will discuss the limitations found during repack, the mitigations put in place, and why separating the scheduler into two backends was necessary to avoid performance issues during CTA operations.

Primary author

Presentation materials