Nov 4 – 8, 2019
Adelaide Convention Centre
Australia/Adelaide timezone

Beyond HEP: Photon and accelerator science computing infrastructure at DESY

Nov 5, 2019, 3:15 PM
Riverbank R7 (Adelaide Convention Centre)

Riverbank R7

Adelaide Convention Centre

Oral Track 7 – Facilities, Clouds and Containers Track 7 – Facilities, Clouds and Containers


Thomas Hartmann (Deutsches Elektronen-Synchrotron (DE))


DESY is one of the largest accelerator laboratories in Europe, developing and operating state of the art accelerators, used to perform fundamental science in the areas of high-energy physics photon science and accelerator development.\newline

While for decades high energy physics has been the most prominent user of the DESY compute, storage and network infrastructure, various scientific areas as science with photons and accelerator development have catched up and are now dominating the demands on the DESY infrastructure resources, with significant consequences for the IT resource provisioning. In this contribution, we will present an overview of the computational, storage and network resources covering the various physics communities on site.\newline

Ranging from HTC batch-like offline processing in the Grid and the interactive user analyses resources in the National Analysis Factory for
the HEP community, to the computing needs of accelerator development or of photon sciences such as PETRA III or the European XFEL. Since DESY co-hosts these experiments and their data taking, their requirements include fast low-latency online processing for data taking and calibration as well as offline processing, thus HPC workloads, that are run on the dedicated {\it Maxwell} HPC cluster.\newline

As all communities face in the coming years significant challenges due to changing environments and increasing data rates, we will discuss how this will reflect in necessary changes to the computing and storage
We will present DESY compute cloud and container orchestration plans as a possible basis for infrastructure and platform services. We will show examples of Jupyter for small scale interactive analysis, as well as its integration into large scale resources such as batch systems or Spark clusters.\newline

To overcome the fragmentation of the various resources for all scientific communities at DESY , we explore how to integrate them into a seamless user experience in an {\it Interdisciplinary Data and Analysis Facility}

Consider for promotion Yes

Primary authors

Thomas Hartmann (Deutsches Elektronen-Synchrotron (DE)) Patrick Fuhrmann Volker Guelzow (Deutsches Elektronen-Synchrotron (DE)) Yves Kemp (Deutsches Elektronen-Synchrotron (DE)) Andreas Gellrich (DESY) Birgit Lewendel (Deutsches Elektronen-Synchrotron (DE)) Dr Christian Voss (DESY) Christoph Beyer Thomas Finnern (DESY) Martin Flemming (DESY) Martin Gasthuber (DESY) Stefan Dietrich (DESY) frank schluenzen Sven Sternberger (DESY) Michael Schuh (Deutsches Elektronen-Synchrotron DESY) Mr Reppin Johannes (DESY) Stefan Bujack Mr Markus Wengert (DESY)

Presentation materials