| Talk | |||||||||||
| Title | EOS as storage back-end for JRC data science | ||||||||||
| Video |
| ||||||||||
| Author(s) | Burger, Armin (speaker) (European Commission - Joint Research Centre) | ||||||||||
| Corporate author(s) | CERN. Geneva | ||||||||||
| Imprint | 2020-02-03. - 0:18:17. | ||||||||||
| Series | (HEP Computing) (EOS workshop) | ||||||||||
| Lecture note | on 2020-02-03T10:45:00 | ||||||||||
| Subject category | HEP Computing | ||||||||||
| Abstract | The Joint Research Centre (JRC) of the European Commission has set up the JRC Earth Observation Data and Processing Platform (JEODPP) as an infrastructure to enable the JRC projects to process and analyze big data, extracting knowledge and insights in support of EU policy making. The main focus is related to geospatial data, but has been extended to other data domains. EOS is the main storage component of the platform, operationally used since mid 2016, maintained and extended with support by the CERN EOS team. The JEODPP is actively used by more than 40 JRC projects as platform for data science, covering a wide range of data analysis activities. In order to serve the growing needs for data storage and processing capacity by the JRC projects, the platform has been extended in 2019. It currently consists of the EOS system as storage back-end with a total gross capacity of 15.5 PB, and service nodes with a total of 2000 CPU cores. As main changes in 2019, the EOS service has been migrated to the QuarkDB namespace, and half of the service nodes have been migrated to FUSEX client. The presentation will give an overview about the implemented platform, the current status, experiences made, and issues identified with EOS as main storage back-end of the JRC data science platform. | ||||||||||
| Copyright/License | © 2020-2024 CERN | ||||||||||
| Submitted by | andreas.joachim.peters@cern.ch | ||||||||||
Record created 2020-02-07, last modified 2022-11-02
Talk details