14-18 October 2013
Amsterdam, Beurs van Berlage
Europe/Amsterdam timezone

The ATLAS EventIndex: an event catalogue for experiments collecting large amounts of data

14 Oct 2013, 15:00
45m
Grote zaal (Amsterdam, Beurs van Berlage)

Grote zaal

Amsterdam, Beurs van Berlage

Poster presentation Data Stores, Data Bases, and Storage Systems Poster presentations

Speaker

Dr Dario Barberis (Università e INFN Genova (IT))

Description

Modern scientific experiments collect vast amounts of data that must be cataloged to meet multiple use cases and search criteria. In particular, high-energy physics experiments currently in operation produce several billion events per year. A database with the references to the files including each event in every stage of processing is necessary in order to retrieve the selected events from data storage systems. The ATLAS EventIndex project is studying the best way to store the necessary information using modern data storage technologies (Hadoop, HBase etc.) that allow saving in memory key-value pairs and select the best tools to support this application from the point of view of performance, robustness and ease of use. At the end of this development, a new technology that is inherently independent of the type of data that are stored in the database -- and therefore directly applicable to all scientific experiments with large amounts of data -- will be available and demonstrated by the example of the ATLAS experiment. This paper describes the initial design and performance tests and the project evolution towards deployment and operation in 2014.

Primary author

Dr Dario Barberis (Università e INFN Genova (IT))

Co-authors

Alvaro Fernandez Casani (Universidad de Valencia (ES)) Dr David Malon (Argonne National Laboratory (US)) Gancho Dimitrov (CERN) Dr JOSE SALT (IFIC-VALENCIA) Dr Jack Cranshaw (Argonne National Laboratory (US)) Javier Sanchez (IFIC) Dr Julius Hrivnac (Universite de Paris-Sud 11 (FR)) Marcin Nowak (Brookhaven National Laboratory (US)) Qizhi Zhang (Argonne National Laboratory (US)) Roman Sorokoletov (University of Texas at Arlington (US)) Dr Santiago Gonzalez De La Hoz (IFIC-Valencia)

Presentation Materials