19-23 May 2014
LAPP
Europe/Paris timezone

Batch system data locality via managed caches

22 May 2014, 09:00
25m
Auditorium Marcel Vivargent (LAPP)

Auditorium Marcel Vivargent

LAPP

9 Chemin de Bellevue 74940 Annecy-le-Vieux FRANCE <b>GPS coordinates :</b> N 45° 55' 14.002'' E 6° 9' 33.998''
Storage & Filesystems Storage and file systems

Speaker

Max Fischer (KIT - Karlsruhe Institute of Technology (DE))

Description

Modern data processing solutions increasingly rely on data locality to achieve high data access rates and scalability. In contrast the common HEP system architectures emphasis uniform resource pools with minimal locality, allowing even for cross-site data access. The concept for the new High Performance Data Analysis (HPDA) Tier3 at KIT aims at introducing data locality to HEP batch systems. Coordinating dedicated cache drives on worker nodes, existing storage hierarchies are extended into the active batch system. The presentation will illustrate the considerations of extending the classic batch architecture and showcase the planned software and hardware architecture of the HPDA T3.

Primary author

Max Fischer (KIT - Karlsruhe Institute of Technology (DE))

Presentation Materials