21-27 March 2009
Prague
Europe/Prague timezone

Lustre File System Evaluation at FNAL

23 Mar 2009, 18:10
20m
Club B (Prague)

Club B

Prague

Prague Congress Centre 5. května 65, 140 00 Prague 4, Czech Republic
oral Hardware and Computing Fabrics Hardware and Computing Fabrics

Speaker

Stephen Wolbers (FNAL)

Description

As part of its mission to provide integrated storage for a variety of experiments and use patterns, Fermilab's Computing Division examines emerging technologies and reevaluates existing ones to identify the storage solutions satisfying stakeholders' requirements, while providing adequate reliability, security, data integrity and maintainability. We formulated a set of criteria and then analyzed several commercial and open-source storage systems. In this paper we present and justify our evaluation criteria, which have two variants, one for HEP event analysis and one for HPC applications as found in LQCD and Computational Cosmology. We then examine in detail Lustre and compare it to dCache, the predominant (by byte count) storage system for LHC data. After a period of testing we released a Lustre system for use by Fermilab's Computational Cosmology cluster in a limited production environment. The Lattice QCD project will prototype a larger Lustre installation on their Infiniband-based clusters. Finally, we discuss Lustre's fitness for the HEP domain and production environments, and the possible integration of Lustre with GridFTP, SRM, and Enstore HSM.

Primary author

Co-authors

Presentation Materials