21-25 May 2012
New York City, NY, USA
US/Eastern timezone

Analysing I/O bottlenecks in LHC data analysis on grid storage resources

22 May 2012, 13:30
25m
Room 914 (Kimmel Center)

Room 914

Kimmel Center

Parallel Computer Facilities, Production Grids and Networking (track 4) Computer Facilities, Production Grids and Networking

Speaker

Wahid Bhimji (University of Edinburgh (GB))

Description

We describe recent I/O testing frameworks that we have developed and applied within the UK GridPP Collaboration, the ATLAS experiment and the DPM team, for a variety of distinct purposes. These include benchmarking vendor supplied storage products, discovering scaling limits of SRM solutions, tuning of storage systems for experiment data analysis, evaluating file access protocols, and exploring IO read patterns of experiment software and their underlying event data models. With multiple grid sites now dealing with petabytes of data, such studies are becoming increasingly essential. We describe how the tests build, and improve, on previous work and contrast how the use-cases differ. We also detail the results obtained and the implications for storage hardware, middleware and experiment software.

Primary author

Wahid Bhimji (University of Edinburgh (GB))

Co-authors

Ilija Vukotic (Universite de Paris-Sud 11 (FR)) Martin Philipp Hellmich (University of Edinburgh) Matthew Doidge (Lancaster University) Philip Clark (University of Edinburgh (GB))

Presentation Materials