CHEP 06

Behind the Scenes Look at Integrity in a Permanent Storage System

15 Feb 2006, 09:00

9h 10m

Tata Institute of Fundamental Research

poster Computing Facilities and Networking Poster

Speaker

Dr Gene Oleynik (Fermilab)

Description

Fermilab provides a primary and tertiary permanent storage facility for its High Energy Physics program and other world wide scientific endeavors. The lifetime of the files in this facility, which are maintained in automated robotic tape libraries, is typically many years. Currently the amount of data in the Fermilab permanent store facility is 3.3 PB and growing rapidly. The Fermilab "enstore" software provides a file system based interface to the permanent store. While access to files through this interface is simple and straightforward, there is a lot that goes on behind the scenes to provide reliable and fast file access, and to insure file integrity and high availability. This paper discusses the measures enstore takes and the administrative steps that are taken to assure users’ files are kept safe, secure, and readily accessible over their long lifetimes. Techniques such as automated write protection, randomized file and tape integrity audits, tape lifetime strategies, and metadata protection are discussed in detail.