14–18 Oct 2013
Amsterdam, Beurs van Berlage
Europe/Amsterdam timezone

A Preview of a Novel Architecture for Large Scale Storage

15 Oct 2013, 14:39
20m
Administratiezaal (Amsterdam, Beurs van Berlage)

Administratiezaal

Amsterdam, Beurs van Berlage

Oral presentation to parallel session Data Stores, Data Bases, and Storage Systems Data Stores, Data Bases, and Storage Systems

Speaker

Andreas Petzold (KIT - Karlsruhe Institute of Technology (DE))

Description

The need for storage continues to grow at a dazzling pace and science and society have become dependent on access to digital data. First sites storing an exabyte of data will be reality in a few years. The common storage technology in small and large computer centers continues to be magnetic disks because of their very good price performance ratio. Storage class memory and solid state disk (ssd) storage is entering the data center but currently foremost in combination with traditional magnetic storage in which it bridges the gap between access time and large storage capacity. The storage building block usually consists of tens to hundreds of magnetic disks connected to a dedicated storage controller that logically aggregates multiple disks into large RAID groups. The storage controller connects to the storage fabric and the application servers via fibre channel, infiniband or tcp/ip and runs proprietary firmware on special purpose hardware. The advent of powerful multicore processors and high bandwidth shared memory of the uniform Intel platform, allows sharing of storage management applications and storage controller ops on the same server. Alternatively the storage management application, normally running on the storage server can run on the storage controller when both have the same architecture. In this presentation we evaluate solutions to be used for large scale storage environments such as the German WLCG Tier 1 center GridKa or the Large Scale Data Facility, hosted by Steinbuch Centre for Computing at Karlsruhe Institute of Technology. On common hardware the storage controller functionality is shared with popular storage management applications like xrootd and dCache. The financial and operational benefits are discussed and first experiences are presented.

Primary author

Andreas Petzold (KIT - Karlsruhe Institute of Technology (DE))

Co-authors

Christoph Erdmann Pfeiler (Forschungszentrum Karlsruhe GmbH (FZK)) Jos Van Wezel (KIT - Karlsruhe Institute of Technology (DE))

Presentation materials