Mar 21 – 27, 2009
Europe/Prague timezone

Lustre File System Evaluation at FNAL

Mar 23, 2009, 6:10 PM
Club B (Prague)

Club B


Prague Congress Centre 5. května 65, 140 00 Prague 4, Czech Republic
oral Hardware and Computing Fabrics Hardware and Computing Fabrics


Stephen Wolbers (FNAL)


As part of its mission to provide integrated storage for a variety of experiments and use patterns, Fermilab's Computing Division examines emerging technologies and reevaluates existing ones to identify the storage solutions satisfying stakeholders' requirements, while providing adequate reliability, security, data integrity and maintainability. We formulated a set of criteria and then analyzed several commercial and open-source storage systems. In this paper we present and justify our evaluation criteria, which have two variants, one for HEP event analysis and one for HPC applications as found in LQCD and Computational Cosmology. We then examine in detail Lustre and compare it to dCache, the predominant (by byte count) storage system for LHC data. After a period of testing we released a Lustre system for use by Fermilab's Computational Cosmology cluster in a limited production environment. The Lattice QCD project will prototype a larger Lustre installation on their Infiniband-based clusters. Finally, we discuss Lustre's fitness for the HEP domain and production environments, and the possible integration of Lustre with GridFTP, SRM, and Enstore HSM.

Primary author


Presentation materials