25–29 May 2026
Chulalongkorn University
Asia/Bangkok timezone

Migration of Conditions data caching from Squid to Varnish in the ATLAS experiment

28 May 2026, 16:51
18m
Chulalongkorn University

Chulalongkorn University

Oral Presentation Track 1 - Data and metadata organization, management and access Track 1 - Data and metadata organization, management and access

Speaker

Ilija Vukotic (University of Chicago (US))

Description

Efficient access to Conditions data is critical for data processing in the ATLAS experiment at the LHC. For more than a decade, Squid HTTP proxies deployed across distributed computing sites have provided low-latency access, reduced WAN bandwidth consumption, and protected origin servers from excessive load. Conditions data traffic is characterized by exceptionally high request rates - often exceeding 20,000 requests per minute at large Tier-2 sites - with cache hit rates above 99.9%. To meet growing performance and operational demands, ATLAS has modernized this infrastructure by adopting Varnish HTTP reverse proxies with RAM-only storage, selected for their high throughput, robustness, and strong community ecosystem. A streamlined deployment strategy was implemented: a small number of high-capacity regional proxies were installed at major sites and placed behind location-aware Cloudflare DNS load balancers, providing seamless global access. Additional local proxies were introduced only where required to meet performance or special needs. The origin server layer - the Frontier launchpads - was re-architected as a high-availability Kubernetes-based service. The new design integrates Nginx load balancers, Varnish proxies, and Tomcat-based Frontier servlets into resilient launchpad clusters. The modernization effort also introduced a comprehensive monitoring and alerting framework, including an AI-driven anomaly-detection agent capable of identifying irregular traffic patterns and issuing proactive alerts. This upgraded architecture significantly improves scalability, resilience, and operational efficiency, ensuring robust delivery of Conditions data for the medium term and providing a model for similar services in support of future LHC runs.

Authors

Alessandro De Salvo Alessandro de salvo (Sapienza Universita e INFN, Roma I (IT)) Andrea Formica (CEA/IRFU,Centre d'etude de Saclay Gif-sur-Yvette (FR)) Andrea Formica (Université Paris-Saclay (FR)) Ilija Vukotic (University of Chicago (US)) Jose Enrique Garcia Navarro (IFIC, CSIC-UV (ES)) Michal Svatos (Czech Academy of Sciences (CZ)) Nurcan Ozturk (University of Texas at Arlington (US))

Presentation materials

There are no materials yet.