Apr 12 – 16, 2010
Uppsala University
Europe/Stockholm timezone

LOFAR Archive Information System

Apr 14, 2010, 2:00 PM
15m
Room IV (Uppsala University)

Room IV

Uppsala University

Oral End-user environments, scientific gateways and portal technologies Astronomy and Astrophysics

Speaker

Fokke Dijkstra (Donald Smits Center for Information Technology, University of Groningen, The Netherlands)

Description

The LOFAR radio telescope, consisting of several stations with antennas, will become operational in spring 2010. The signals are collected and correlated by a central supercomputer. The resulting data, several petabytes a year, will be stored in an archive, distributed over a number of partners. The LOFAR Archive Information System plays a crucial role in this long-term archive. It will keep track of the data stored in the archive, and allows for further processing of that data. The information system supports different computing and storage technologies, including the EGEE Grid.

Detailed analysis

The LOFAR Archive Information System is a cooperation between OmegaCEN/RUG, Big Grid and Target, and built on the basis of the Astro-WISE Information System. It supports different storage technologies, like the Astro-WISE data-server, SRM storage and the Target storage system. It not only keeps track of where the data is stored but also keeps track of the processing workflow, which was used for the data processing. Both the processing parameters, including software versions, and relevant intermediate results are stored. Processing jobs can be submitted to the EGEE Grid.
Much work has been performed on the integration of Astro-WISE with the Grid. Special attention has been paid to the integration of 3rd party java versions of Grid client software with Astro-WISE. This, to make the environment as user friendly as possible, without requiring a full gLite installation for all clients. For transferring proxy credentials to the job submission engine a MyProxy server is used.
The different storage interfaces are hidden from the user by making use of the object oriented approach of Astro-WISE. The proper storage interface to be used for the data is therefore instantiated automatically.

Impact

The construction of the LOFAR long-term archive is fundamentally important for astronomers who want to work with the LOFAR data. This archive will store petabytes of data, and will include processing capabilities. The LOFAR Archive Information System is completely metadata oriented. The full data lineage can be stored for the data that is processed, and data can be processed on the fly. Furthermore the Astro-WISE system is very flexible and gives the astronomers powerful tools to analyse their data, using a common approach and a common data model. All of this has been connected to the EGEE Grid as one of the suppliers of storage and computing.
What differentiates the LOFAR information system from other Storage Grid environments is the key role for the metadata, the full data lineage, and the target-processing capability, where processing is started when necessary.

Conclusions and Future Work

The LOFAR Archive Information System is a key component of the LOFAR long-term archive, enabling researchers to find and process LOFAR data, while keeping track of the data lineage.
Future work will focus on further refinement of the processing and storage capabilities, using input obtained during processing and service challenges. To allow general use of the archive, without requiring end-user certificates, the use of robot certificates is planned for services that allow retrieving data and starting up processing. This also allows immediate coupling with the Virtual Observatory.

URL for further information http://www.astro-wise.org/Lofar.html
Keywords Astronomy, LOFAR, Data management, metadata, processing, Grid

Authors

Andrey Belikov (Kapteyn Astronomical Institute, University of Groningen, The Netherlands) Fokke Dijkstra (Donald Smits Center for Information Technology, University of Groningen, The Netherlands) Willem-Jan Vriend (Kapteyn Astronomical Institute, University of Groningen, The Netherlands) Zheng Meyer-Zhao (Donald Smits Center for Information Technology, University of Groningen, The Netherlands)

Co-authors

Adriaan Renting (Astron, The Netherlands) Danny Boxhoorn (Kapteyn Astronomical Institute, University of Groningen, The Netherlands) Dennis Van Dok (Nikhef, The Netherlands) Edwin Valentijn (Kapteyn Astronomical Institute, University of Groningen, The Netherlands) Floris Sluiter (SARA, The Netherlands) Hanno Holties (Astron, The Netherlands) Kor Begeman (Kapteyn Astronomical Institute, University of Groningen, The Netherlands)

Presentation materials