10–14 Oct 2016
San Francisco Marriott Marquis
America/Los_Angeles timezone

Efficient Access to Massive Amounts of Tape-Resident Data

13 Oct 2016, 12:15
15m
Sierra B (San Francisco Mariott Marquis)

Sierra B

San Francisco Mariott Marquis

Oral Track 6: Infrastructures Track 6: Infrastructures

Speaker

David Yu (Brookhaven National Laboratory (US))

Description

Randomly restoring files from tapes degrades the read performance primarily due to frequent tape mounts. The high latency and time-consuming tape mount and dismount is a major issue when accessing massive amounts of data from tape storage. BNL's mass storage system currently holds more than 80 PB of data on tapes, managed by HPSS. To restore files from HPSS, we make use of a scheduler software, called ERADAT. This scheduler system was originally based on code from Oak Ridge National Lab, developed in the early 2000s. After some major modifications and enhancements, ERADAT now provides advanced HPSS resource management, priority queuing, resource sharing, web-browser visibility of real-time staging activities and advanced real-time statistics and graphs. ERADAT is also integrated with ACSLS and HPSS for near real-time mount statistics and resource control in HPSS. ERADAT is also the interface between HPSS and other applications such as the locally developed Data Carousel providing fair resource-sharing policies and related capabilities.
ERADAT has demonstrated great performance at BNL and other scientific organizations.

Primary Keyword (Mandatory) Software development process and tools
Secondary Keyword (Optional) Storage systems

Primary author

David Yu (Brookhaven National Laboratory (US))

Co-author

Dr Jerome LAURET (Brookhaven National Laboratory)

Presentation materials