17 September 2019
CERN
Europe/Zurich timezone

Ad-hoc filesystems for dynamic science workloads

17 Sept 2019, 14:25
25m
500/1-001 - Main Auditorium (CERN)

500/1-001 - Main Auditorium

CERN

46.232615, 6.045141
400
Show room on map

Speakers

Mr Stig Telfer (StackHPC Ltd)Mr John Garbutt (StackHPC Ltd)

Description

In this talk, we present recent work supporting the computing demands of the Euclid space mission using resources from across an OpenStack federation for scientific computing. CephFS is used to present a single coherent filesystem drawing upon resources from multiple sites. We present our approach and experiences.

We then present an alternative approach to filesystems on-the-fly using the Data Accelerator project at Cambridge University, currently #1 in the global IO-500 list. We provide an overview of the technologies involved and an analysis of how its high performance levels are achieved.

Authors

Mr Stig Telfer (StackHPC Ltd) Mr John Garbutt (StackHPC Ltd)

Presentation materials