14-16 September 2015
University "La Sapienza", Physics Department
Europe/Rome timezone

Using IKAROS as a data transfer and management utility within the KM3NeT computing model

15 Sep 2015, 15:45
15m
Aula Rasetti, 2nd floor (University "La Sapienza", Physics Department)

Aula Rasetti, 2nd floor

University "La Sapienza", Physics Department

Piazzale Aldo Moro, 2 - 00185 Roma - Italy

Speaker

Christos Filippidis (Nat. Cent. for Sci. Res. Demokritos (GR))

Description

International collaborative scientific experiments are generating datasets which are increasing exponentially in both complexity and volume, making their analysis, archival, and sharing one of the grand challenges of the 21st century. These experiments, in their majority, adopt computing models consisting of different Tiers (each Tier is made up of several computing centres and provides a specific set of services) and for the different steps of data processing (simulation, filtering, calibration, reconstruction and analysis) several software packages are utilized. The computing requirements are extremely demanding and, usually, span from serial to multi-parallel or GPU-optimized jobs. The collaborative nature of these experiments demands very frequent WAN data transfers and data sharing among individuals and groups. Typically, such a computing model utilizes several different computing infrastructures like: Grids, Clouds, HPCs, Data Centers and Local computing Clusters. IKAROS is a framework that enables us to create scalable storage formations on-demand and helps us address several limitations which the current file systems face when dealing with very large scale infrastructures. It enables us to create ad-hoc nearby storage formations and can use a huge number of I/O nodes in order to increase the available bandwidth (I/O and network). IKAROS unifies remote and local access in the overall data flow, by permitting direct access to each I/O node. In this way we can handle the overall data flow at the network layer, limiting in this way the interaction with the operating system. This approach enables us to virtually connect, at the users level, the several different computing facilities used (Grids, Clouds, HPCs, Data Centers, Local computing Clusters and personal storage devices), on-demand, based on the needs, by using well known standards and protocols, like HTTP.

Primary author

Christos Filippidis (Nat. Cent. for Sci. Res. Demokritos (GR))

Co-authors

Christos Markou (Institute of Nuclear Physics) Prof. Yiannis Cotronis (University of Athens)

Presentation Materials