21-25 May 2012
New York City, NY, USA
US/Eastern timezone

Development of noSQL data storage for the ATLAS PanDA Monitoring System

22 May 2012, 13:30
4h 45m
Rosenthal Pavilion (10th floor) (Kimmel Center)

Rosenthal Pavilion (10th floor)

Kimmel Center

Poster Distributed Processing and Analysis on Grids and Clouds (track 3) Poster Session

Speaker

Maxim Potekhin (Brookhaven National Laboratory (US))

Description

For several years the PanDA Workload Management System has been the basis for distributed production and analysis for the ATLAS experiment at the LHC. Since the start of data taking PanDA usage has ramped up steadily, typically exceeding 500k completed jobs/day by June 2011. The associated monitoring data volume has been rising as well, to levels that present a new set of challenges in the areas of database scalability and monitoring system performance and efficiency. These challenges have being met with a R&D and development effort aimed at implementing a scalable and efficient monitoring data storage based on a noSQL solution (Cassandra). We present the data design and indexing strategies for efficient queries, as well as our experience of operating a Cassandra cluster and interfacing it with a Web service.

Primary author

Co-authors

Hironori Ito (Brookhaven National Laboratory (US)) Maxim Potekhin (Brookhaven National Laboratory (US)) Dr Torre Wenaus (Brookhaven National Laboratory (US))

Presentation Materials

There are no materials yet.