Oct 17 – 21, 2016
LBNL
US/Pacific timezone

Effective Data Retrieval from Massive Amounts of Tape-Resident Data

Oct 19, 2016, 4:10 PM
25m
Building 50 Auditorium (LBNL)

Building 50 Auditorium

LBNL

Berkeley, CA 94720
Storage & Filesystems Storage and Filesystems

Speaker

David Yu (Brookhaven National Laboratory (US))

Description

Randomly restoring files from tapes degrades the read performance primarily due to frequent tape mounts. The high latency and time-consuming tape mount and dismount is a major issue when accessing massive amounts of data from tape storage. BNL's mass storage system currently holds more than 80 PB of data on tapes, managed by HPSS. To restore files from HPSS, we make use of a scheduler software, called ERADAT. This scheduler system was originally based on code from Oak Ridge National Lab, developed in the early 2000s. After some major modifications and enhancements, ERADAT now provides advanced HPSS resource management, priority queuing, resource sharing, web-browser visibility of real-time staging activities and advanced real-time statistics and graphs. ERADAT is also integrated with ACSLS and HPSS for near real-time mount statistics and resource control in HPSS. ERADAT is also the interface between HPSS and other applications such as the locally developed Data Carousel providing fair resource-sharing policies and related capabilities.
ERADAT has demonstrated great performance at BNL and other scientific organizations.

Primary author

David Yu (Brookhaven National Laboratory (US))

Presentation materials