5th EGEE User Forum

Migration of the MAGIC data transfers to a Grid infrastructure

12 Apr 2010, 17:21

3m

Aula (Uppsala University)

Poster Experiences from application porting and deployment Poster session

Speaker

Roger Firpo Curcoll (Port d'Informació Científica)

Description

The MAGIC collaboration is moving from a computing model based on local computer farms to a Grid based model, including the migration of all the services of the official data center. Here we present the recent progress in the adoption of the Grid infrastructure in the MAGIC data center, which relates to the data transfer from the observation site.

Conclusions and Future Work

The MAGIC data transfer from the observatory to the data center has been migrated to Grid. This is a further step of the data center into Grid after the migration of the storage, computing and data access services. The next step in this process will be to develop high level analysis tools for users to better use the datacenter services.

Impact

The computing system in the MAGIC observation site was not initially designed to interact with Grid, and data transfer and publishing to Grid were made through ssh connections and an intermediate system in the data center. In the last months we have installed a new server to publish the data in the observation site through a SRM endpoint, and also ported all our data transfer tools to use this infrastructure. This allowed us to save resources and reduce the complexity of the system. Our initial tests with the new infrastructure also point to a better use of the available bandwidth, which can lead to deprecating the tape-based transfer of raw data.

Detailed analysis

The MAGIC (Major Atmospheric Gamma-ray Imaging Cherenkov) telescopes, run by an international collaboration of institutes from 9 different countries, is a two 17-meter Cherenkov telescope system located on La Palma (Canary Islands) dedicated to the study of the universe in very high energy gamma-rays. The addition of the second telescope caused the data output of the system to rise up to 300 TB per year of data. The MAGIC data center hosted by the Port d'Informaci— Cient’fica (PIC) in Barcelona is migrating its services to Grid as part of an upgrade needed to deal with the increased data volume. After migrating the data to a Grid filesystem, we have ported the data transfer service. In the last months we have set up an SRM endpoint in the observation site and ported all the data transfer tools to use Grid file transfers.