Speaker
Dr
Jörg Meyer
(KIT - Karlsruher Institute of Technology)
Description
After analysis and publication, there is no need to keep experimental data online on spinning disks. For reliability and costs inactive data is moved to tape and put into a data archive. The data archive must provide reliable access for at least ten years following a recommendation of the German Science Foundation (DFG), but many scientific communities wish to keep data available much longer. Data archival is on the one hand purely a bit preservation activity in order to ensure the bits read are the same as those written years before. On the other hand enough information must be archived to be able to use and interpretate the content of the data. The latter is depending on many also community specific factors and remains an areas of much debate among archival specialists.
The paper describes the current practice of archival and bit preservation in use for different science communities at KIT for which a combination of organisational services and technical tools are required. The special monitoring to detect tape related errors, the software infrastructure in use as well as the service certification are discussed. Plans and developments at KIT also in the context of the Large Scale Data Management and Analysis (LSDMA) project are presented. The technical advantages of the T10 SCSI Stream Commands (SSC-4) and the Linear Tape File System (LTFS) will have a profound impact on future long term archival of large data sets.
Primary author
Dr
Jörg Meyer
(KIT - Karlsruher Institute of Technology)
Co-authors
Achim Streit
(KIT - Karlsruhe Institute of Technology (DE))
Jos Van Wezel
(KIT - Karlsruhe Institute of Technology (DE))
Marcus Hardt
(Kalrsruhe Institute of Technology)
