14-18 October 2013
Amsterdam, Beurs van Berlage
Europe/Amsterdam timezone

Popularity Prediction Tool for ATLAS Distributed Data Management

14 Oct 2013, 15:00
45m
Grote zaal (Amsterdam, Beurs van Berlage)

Grote zaal

Amsterdam, Beurs van Berlage

Poster presentation Data Stores, Data Bases, and Storage Systems Poster presentations

Speaker

Graeme Andrew Stewart (CERN)

Description

This paper describes a popularity prediction tool for data-intensive data management systems, such as the ATLAS distributed data management (DDM) system. The tool is fed by the DDM popularity system, which produces historical reports about ATLAS data usage and provides information about the files, datasets, users and sites where data was accessed. The tool described in this contribution uses this historic information to make a prediction about the future popularity of data. It finds trends in the usage of data using a set of neural networks and a set of input parameters and predicts the number of accesses in the near term future. This information can then be used in a second step to improve the distribution of replicas at sites, taking into account the cost of creating new replicas (bandwidth and load on the storage system) compared to the gain of having new ones (faster access of data for analysis). The tool ensures that the total amount of space available on the grid is not exceeded. This information can then help to make a decision about adding and also removing data from the grid to make a better use of the available resources. The design and architecture of the popularity prediction tool is described, examples of its use are shown and an evaluation of its performance is presented.

Primary author

Thomas Beermann (Bergische Universitaet Wuppertal (DE))

Co-authors

Angelos Molfetas (University of Sydney (AU)) Armin Nairz (CERN) Cedric Serfon (CERN) Erich Schikuta (University of Vienna) Graeme Andrew Stewart (CERN) Dr Luc Goossens (CERN) Mario Lassnig (CERN) Martin Barisits (CERN) Ralph Vigne (University of Vienna (AT)) Vincent Garonne (CERN)

Presentation Materials