Speaker
Dr
Harald Kornmayer
(FORSCHUNGSZENTRUM KARLSRUHE (FZK))
Description
Application context and scientific goals
========================================
The field of gamma-ray observations in the energy range between 10 GeV
and 10 TeV developed fast over the last decade.
From the first observation of TeV gamma rays from the Crab nebula using the
atmospheric Cerenkov imaging technique in 1989 [1] to the
discovery of new gamma ray sources with the new generation telescopes
like the HESS observation of a high-energy particle acceleration
in the shell of a supernova remnant [2], a
new observation window to the universe was opened.
In the future other ground based VHE $\gamma$-ray observatories
(namely MAGIC [3], VERITAS [4]
and KANGAROO [5]) will significantly
contribute to the exploitation of this new observation window.
With the new generation Cerenkov telescopes the requirements for the
analysis and Monte Carlo production computing infrastructure
will increase due to a higher number of camera pixels,
faster FADC systems and a bigger mirror size.
In the future the impact of VHE gamma-ray astronomy
will increase by joined observations of different Cerenkov telescopes.
In 2003 the national Grid centers in Italy (CNAF), Spain (PIC) and Germany (GridKA)
started together with the MAGIC collaboration an effort to build a
distributed computing system for Monte Carlo generation and analysis on top of existing
Grid infrastructure.
The MAGIC telescope was chosen due to the following reasons:
o The MAGIC collaboration is international, with most partners from Europe
o main partners of the MAGIC telescope are located close to the national Grid centers
o The generation of Monte Carlo data is very compute intensive, specially to get
enough statistics
in the low energy range.
o The analysis of the fast increasing real data samples will be done in different
institutes. The collaborators need a seamless access to the data while reducing the
number of
replicas to a minimum.
o The MAGIC collaboration will build a second telescope in 2007 resulting in a
doubled data rate.
The idea of the MAGIC Grid [6] was presented to the EGEE Generic
Application Advisory Panel (EGAAP).
In June 2004 EGEE accepted the generation of Monte Carlo data for the MAGIC
telescope as one of the generic applications of the project.
Grid added value
================
By implementing the MAGIC Grid over the last two years, the MAGIC collaboration
benefit in many aspects. These aspects are described in this chapter.
o Collaboration of different institutes
By combining the resources of the MAGIC collaborators and the reliable
resources from the national Grid centers the MAGIC collaborators
will be empowered to use their computing infrastructure more efficiently.
The time to analyse the big amount of data to solve
specific scientific problems will be shortend.
o Cost reduction
By using the EGEE infrastructure and the EGEE services the effort for
MAGIC collaboration to build a distributed computing system for
the Monte Carlo simulations was significantly reduced.
o Speedup of Monte Carlo production
As the MAGIC Monte Carlo System was build on top of the EGEE middleware
the integration of new computing resources is very easy. By getting
support from many different EGEE resource providers the production
rate for the Monte Carlos can be increased very easily.
o Persistent storage of observation data
The MAGIC telescope will produce a lot of data in the future. These
data are currently stored on local resources including disk systems
and tape libraries. The MAGIC collaboration recognized that this
effort is not negligible especially concerning man power. Therefore
the observation data will be stored by the spanish Grid center PIC.
o Data availability improvements
By importing the observation data to the Grid, the MAGIC
collaboration expect that the availablitly of data will be
increased with the help of Grid data management methods like
data replication, etc. As the main data services will be provided
in the future by the national Grid centers instead of research university
groups at universities, the overall data availablitly is
expected to increase.
o Cost reduction
By using the EGEE infrastructure and the EGEE services the effort for
MAGIC collaboration to build a distributed computing system for
the Monte Carlo simulations was significantly reduced.
Experiences with the EGEE infrastructure
========================================
The experiences of the developers during the different phases of the
realisation of the MAGIC Monte Carlo production system on the EGEE
Grid infrastructure are described in this chapter. As the MAGIC virtual
organisation was accepted as one of the first generic EGEE application,
the development process was influenced by general developments within the EGEE
project too like changed in the middleware versions, etc.
o Prototype implementation
--------------------------
The migration of the compute intensive MMCS program from a local batch
system to the Grid was done by the definition of a template JDL form.
This template sends all needed input data together with the executable
to the Grid. The resources are chosen by the resource broker.
The automatic registration of the output file as a logical file on the
Grid was not very reliable at the beginning, but improved to production
level within the EGEE project duration.
o Production MAGIC Grid system
------------------------------
The submission of many production system needed the implementation of a
graphical user interface and a database for metadata. The graphical
user interface was realised with the JAVA programming language. The
execution of the LCG/gLite commands is wrapped in JAVA shell commands.
A MySQL database holds the schema for the metadata.
As mentioned above the "copy and register" process for the output file was
not realiable enough an additional job status "DONE (data available)" was
invented. With the help of the database, jobs that did not reach this
job status within two days are resubmitted. The job data are keeped in
a seperate database table to analyse them later.
o Reliability of EGEE services
------------------------------
The general services like resource brokers, VO management tools and Grid
user support was provided by the EGEE resources providers. The MAGIC Grid
is setup on top of this services. A short report of the experiences with
this production services will be given.
Key issues for the future of Grid technology
============================================
The MAGIC collaboration is currently evaluating the EGEE Grid infrastructure
as the backbone for a distributed computing system in the future including
the data storage on Grid data centers like PIC. Furthermore the discussion
with other projects like the HESS collaboration has started
to move towards "Virtual Very High energetic Gamma ray observatory" [7].
The problems and challenges that needs to be solved on the track to a sustainable
Grid infrastructure will be discussed from the user perspective
References:
[1] T. Weekes et al., The Astrophysical Journal, volume 342 (1989), p. 379
[2] F. A. Aharonian et al., Nature 432, 75 - 77 (04 November 2004)
[3] E. Lorenz, 1995, Fruehsjahrtagung Deutsche Physikalische Gesellschaft, March 9-10
[4] T. Weekes et al., Astropart. Phys., 17, 221-243 (2002)
[5] Enomoto, R. et al., Astropart. Phys. 16, 235-244 (2002)
[6] H. Kornmayer et al., "A distributed, Grid-based analysis system for the MAGIC
telescope",
Proceedings of the CHEP Conference , Interlaken, Switzerland, 2004
[7] H. Kornmayer et al., "Towards a virtual observatory for high energetic gamma
rays", Cherenkov 2005,
Paris, 2005
Author
Dr
Harald Kornmayer
(FORSCHUNGSZENTRUM KARLSRUHE (FZK))
Co-authors
Andreu Pacheco
(PIC)
Ariel Garcia
(FZK)
Ciro Bigongiari
(INFN)
Esther Accion
(PIC)
Gonzalo Merino
(PIC)
Jose A,. Coarasa
(MPPMU)
Manuel Delfino
(PIC)
Mirco Mazzucato
(CNAF/INFN)
