Nov 4 – 8, 2019
Adelaide Convention Centre
Australia/Adelaide timezone

Analyzing storage access data with Apache-Spark and Jupiter notebooks

Nov 7, 2019, 3:30 PM
1h
Hall F (Adelaide Convention Centre)

Hall F

Adelaide Convention Centre

Poster Track 7 – Facilities, Clouds and Containers Posters

Speaker

Thomas Hartmann (Deutsches Elektronen-Synchrotron (DE))

Description

Running a data center is never a trivial job. In addition to daily
routine tasks, service operation teams have to provide a meaningful
information for monitoring, reporting and access pattern analytic.
The dCache production instances at DESY, produce gigabytes of billing
files per day. However, with a help of modern BigData analysis tools
like Apache-Spark and Jupiter notebooks such task can be easily achieved.
Moreover, the tool set for storage access analysts can be shared with
scientific community making it re-usable computational resource as well
as shared knowledge

Consider for promotion No

Primary authors

Mr Tigran Mkrtchyan (DESY) Marina Sahakyan Birgit Lewendel (Deutsches Elektronen-Synchrotron (DE)) Dr Christian Voss (DESY) Thomas Hartmann (Deutsches Elektronen-Synchrotron (DE))

Presentation materials