Speaker
Dr
Chih-Hao Huang
(Fermi National Accelerator Laboratory)
Description
ENSTORE is a very successful petabyte-scale mass storage system developed at
Fermilab. Since its inception in the late 1990s, ENSTORE has been serving the
Fermilab community, as well as its collaborators, and now holds more than 3 petabytes
of data on tape. New data is arriving at an ever increasing rate. One practical issue
that we are confronted with is: storage technologies have been evolving at an ever
faster pace. New drives and media have been brought to the market constantly with
larger capacity, better performance, and lower price. It is not cost effective for a
forward looking system to stick with older technologies. In order to keep up with
this technological evolution, ENSTORE was in need of a mechanism to migrate data onto
newer media. Migrating large quantities of data in a highly available mass storage
system does present a technical challenge. An auto-migration scheme was developed in
ENSTORE that carries out this task seamlessly, behind the scenes, and without
interrupting service nor requiring much operational attention. After two years in
service, auto-migration has lived up to its expectation and ENSTORE has gone through
several generations of drives and media. In addition, migration can be used in media
copying, media consolidation, and data compaction. In this paper, we are going to
present the conceptual design of ENSTORE, the issues in data migration in a highly
available mass storage system, the implementation of auto-migration in ENSTORE, our
experience and extended applications.
Primary author
Dr
Chih-Hao Huang
(Fermi National Accelerator Laboratory)
Co-authors
Dr
Alex Moibenko
(Fermi National Accelerator Laboratory)
Mr
David Berg
(Fermi National Accelerator Laboratory)
Dr
Dmitry Litvintsev
(Fermi National Accelerator Laboratory)
Mr
Don Petravick
(Fermi National Accelerator Laboratory)
Ms
Eileen Berman
(Fermi National Accelerator Laboratory)
Mr
Gene Oleynik
(Fermi National Accelerator Laboratory)
Mr
George Szmuksta
(Fermi National Accelerator Laboratory)
Dr
Jon Bakken
(Fermi National Accelerator Laboratory)
Mr
Michael Zalokar
(Fermi National Accelerator Laboratory)
Mr
Terry Jones
(Fermi National Accelerator Laboratory)
Mr
Wayne Baisley
(Fermi National Accelerator Laboratory)
