EOS Workshop

Session

Using EOS

5 Feb 2018, 13:45

31/3-004 - IT Amphitheatre (CERN)

13. EOS as a Data Lake technology

Xavier Espinal Curull (CERN)

05/02/2018, 13:45

The computing strategy document for HL-LHC identifies storage as one of the main WLCG challenges in one decade from now. In the naive assumption of applying today¹s computing model, the ATLAS and CMS experiments will need one order of magnitude more storage resources than what could be realistically provided by the funding agencies at the same cost of today. The evolution of the computing...

7. The eXtreme Data Cloud (XDC) Project

Oliver Keeble (CERN)

05/02/2018, 14:05

EOS is participating in the EU-funded eXtreme Data Cloud (XDC) Project which will support work on distributed deployment, caching and federation. This contribution gives an overview of the project and EOS's role within it.

26. O2 Disk Buffer - WP15 EOS Performance Testing Framework

Pete Eby (Oak Ridge National Laboratory - (US))

05/02/2018, 14:20

The ALICE Online/Offline (O2) Disk Buffer project will deploy a 60PB EOS filesystem at CERN to accommodate the Pb-Pb data taking period planned for 2020. An initial ~6PB evaluation system is planned for deployment in May 2018.

Members from CERN, Oak Ridge National Lab (ORNL), and Lawrence Berkeley National Lab (LBNL) are collaborating on Work Package 15 (WP15) in the development of a...

8. EOS Ops at CERN:

Herve Rousseau (CERN)

05/02/2018, 14:40

The EOS operations team at CERN operates multiple instances of EOS for the physics experiments and other activities from the laboratory.

In this presentation we will focus on infrastructure changes, best practices and evolution. A second part will mention the upgrade process we're going through to run Citrine, as well as tools we wrote and use to manage our EOS instances. We will end the talk...

15. EOS as storage back-end for geospatial data analysis

Armin Burger (European Commission - Joint Research Centre), Veselin Vasilev (European Commission, Joint Research Centre (JRC))

05/02/2018, 15:05

The Joint Research Centre (JRC) of the European Commission has set up the JRC Earth Observation Data and Processing Platform (JEODPP) as a pilot infrastructure to enable the knowledge production Units to process and analyze big geospatial data in support to EU policy needs. This platform is built upon commodity hardware and first operational services were made available mid 2016. It currently...

4. EOS at the Fermilab LPC Physics Center

Dan Szkola (Fermi National Accelerator Lab. (US))

05/02/2018, 15:45

We report on operational experiences and future plans with the Fermilab LHC Physics Center (LPC) computing cluster. The LPC cluster is a 4500-core user analysis cluster with 5 PB of storage running EOS. The LPC cluster supports several hundred users annually, from CMS university groups across the US. We anticipate the total EOS storage pool to grow by 50% by the start of Run 3 of the LHC.

3. The Adventures of AARNet Across the EOS Dimension

David Jericho (AARNet)

05/02/2018, 16:05

AARNet's use of EOS for both our production CDN and our CloudStor platform over the last two years has been an adventure in collaboration, experiencing bugs, and extracting esoteric knowledge from both people and the code base.

EOS exists in a space that isn't met by any existing open source scale out storage solutions. Neither Ceph, or any of the less common scale out systems provide the...

27. EOS status of IHEP site

Haibo li (Institute of High Energy Physics Chinese Academy of Science)

05/02/2018, 16:25

This report will talk about the current status and recent updates of EOS at IHEP Site since the first EOS workshop in 2017, covering storage expansion, issues encountered and other related work.

25. T2 Experiences Scaling EOS: Capacity and Performance Observations

Pete Eby (Oak Ridge National Laboratory - (US))

05/02/2018, 16:45

During the last two years Oak Ridge National Laboratory (ORNL) has administered the ORNL::EOS T2 site which has seen two storage capacity expansions with installed capacity increasing from 1PB to 2.5PB. As utilization and capacity have grown observations on the performance impact of underlaying storage architecture, RAID size, filesystem design decisions, and performance tunings have been...

19. New CI platform for EOS and XrootD

Jozsef Makai (CERN)

06/02/2018, 10:30

In the past year, we have migrated the continuous integration platform of EOS, XrootD and all related projects from Jenkins to Gitlab CI in order to provide a more agile, satisfying and all-automated build environment.

Numerous achievements have been reached during the year.

We have introduced builds and packages for new platforms. For EOS, we have created an all-inclusive dmg package for...

2. Down the Rabbit Hole: Adventures in Bug-Hunting

Crystal Chua (AARNet)

06/02/2018, 10:45

This talk covers a journey through fuzz-testing CERN's EOS file system with AFL, from compiling EOS with afl-gcc/afl-g++, to learning to use AFL, and finally, making sense of the results obtained.

Fuzzing is a software testing process that aims to find bugs, and subsequently potential security vulnerabilities, by attempting to trigger unexpected behaviour with random inputs. It is...

1. Boxed: Docker-based service deployment in private and public clouds

Enrico Bocchi (CERN)

06/02/2018, 11:05

Docker containers are rapidly becoming the preferred way to distribute, deploy, and run services by developers and system administrators. Their popularity is rapidly increasing as they constitute an appealing alternative to virtual machines: Containers require a negligible amount of time to set-up, provide performance comparable to the one of the host, and are easy to manage, replicate, and...

10. EOS storage at ALICE using docker

Martin Vala (Technical University of Kosice (SK))

06/02/2018, 11:25

Talk will present new automatic tool to configure/update EOS storage using docker (eos-docker-utils). Currently plain EOS and ALICE EOS storage configurations are supported. First production storage is running for ALICE experiment (ALICE:Kosice::EOS).

16. Demo: Setting up QuarkDB

Georgios Bitzes (CERN)

06/02/2018, 11:45

During this presentation, we will demo the setup and operation of a highly-available QuarkDB cluster, ready to be used as backend for the new EOS namespace.

11. CERN Tape Archive Update

Michael Davis (CERN)

06/02/2018, 14:00

The CERN Tape Archive (CTA) is the tape archival back-end for EOS and the successor to CASTOR. This talk will give an update on CTA developments since last year's EOS workshop.

9. Building Client-Server APIs using the XRootD Scalable Service Interface

Michael Davis (CERN)

06/02/2018, 14:15

This talk will give an overview of the XRootD Scalable Service Interface (SSI), which provides an asynchronous request-response framework with an emphasis on efficient data transfers. This will include a case study explaining how we used SSI and Google Protocol Buffers to develop the API between EOS and the CERN Tape Archive (CTA). The SSI-Protobuf bindings are available as a generic framework...

28. Extensions for policy driven data management

Andreas Joachim Peters (CERN)

06/02/2018, 14:45

In this presentation we will briefly explain the foreseen developments to implement the XDC and data lake concepts.

18. CERNBox: the CERN cloud storage driven by EOS

Luca Mascetti (CERN), Hugo Gonzalez Labrador (CERN)

06/02/2018, 14:55

CERNBox is a the CERN cloud storage service. It allows synchronising and sharing files on all major desktop and mobile platforms (Linux, Windows, MacOSX, Android, iOS) aiming to provide universal access and offline availability to any data stored in the CERN EOS infrastructure.
With more than 12k users registered in the system, CERNBox has responded to the high demand in our diverse community...

30. Experience of deployment of CERNbox_SWAN at SPbSU

Andrey Zarochentsev (St Petersburg State University (RU))

06/02/2018, 15:15

6. CERNBox sharing reloaded: graceful coexistance of POSIX and sync/share in EOS ACLs

Jakub Moscicki (CERN)

06/02/2018, 15:25

I'll discuss possible improvements of the EOS permission system to gracefully support ACLS for both sync/share access (CERNBox) and filesystem access (POSIX). This will also include an implementation of automatic synchronization of the shares from EOS.

This is to address functional shortcoming for current CERNBox users and prepare for future massive filesystem access to EOS user instances at CERN.

29. A Microservice Architecture for CERNBOX

Hugo Gonzalez Labrador (CERN)

06/02/2018, 15:40