21-25 May 2012
New York City, NY, USA
US/Eastern timezone

Improving ATLAS grid site reliability with functional tests using HammerCloud

22 May 2012, 13:30
4h 45m
Rosenthal Pavilion (10th floor) (Kimmel Center)

Rosenthal Pavilion (10th floor)

Kimmel Center

Poster Distributed Processing and Analysis on Grids and Clouds (track 3) Poster Session

Speaker

Federica Legger (Ludwig-Maximilians-Univ. Muenchen)

Description

With the exponential growth of LHC (Large Hadron Collider) data in 2011, and more to come in 2012, distributed computing has become the established way to analyse collider data. The ATLAS grid infrastructure includes more than 80 sites worldwide, ranging from large national computing centers to smaller university clusters. These facilities are used for data reconstruction and simulation, which are centrally managed by the ATLAS production system, and for distributed user analysis. To ensure the smooth operation of such a complex system, regular tests of all sites are necessary to validate the site capability of successfully executing user and production jobs. We report on the development, optimization and results of an automated functional testing suite using the HammerCloud framework. Functional tests are short light- weight applications covering typical user analysis and production schemes, which are periodically submitted to all ATLAS grid sites. Results from those tests are collected and used to evaluate site performances. Sites that fail or are unable to run the tests are automatically excluded from the PanDA brokerage system, therefore avoiding user or production jobs to be sent to problematic sites. We show that stricter exclusion policies help to increase the grid reliability, and the percentage of user and production jobs aborted due to network or storage failures can be sensibly reduced using such a system.

Primary author

Co-authors

Daniel Colin Van Der Ster (CERN) Federica Legger (Ludwig-Maximilians-Univ. Muenchen) Dr Gianfranco Sciacca (Universitaet Bern (CH)) Johannes Elmsheuser (Ludwig-Maximilians-Univ. Muenchen (DE)) Ramon Medrano Llamas (Universidad de Oviedo (ES))

Presentation Materials

There are no materials yet.