Speaker
Description
Detailed analysis
The LOFAR Archive Information System is a cooperation between OmegaCEN/RUG, Big Grid and Target, and built on the basis of the Astro-WISE Information System. It supports different storage technologies, like the Astro-WISE data-server, SRM storage and the Target storage system. It not only keeps track of where the data is stored but also keeps track of the processing workflow, which was used for the data processing. Both the processing parameters, including software versions, and relevant intermediate results are stored. Processing jobs can be submitted to the EGEE Grid.
Much work has been performed on the integration of Astro-WISE with the Grid. Special attention has been paid to the integration of 3rd party java versions of Grid client software with Astro-WISE. This, to make the environment as user friendly as possible, without requiring a full gLite installation for all clients. For transferring proxy credentials to the job submission engine a MyProxy server is used.
The different storage interfaces are hidden from the user by making use of the object oriented approach of Astro-WISE. The proper storage interface to be used for the data is therefore instantiated automatically.
Impact
The construction of the LOFAR long-term archive is fundamentally important for astronomers who want to work with the LOFAR data. This archive will store petabytes of data, and will include processing capabilities. The LOFAR Archive Information System is completely metadata oriented. The full data lineage can be stored for the data that is processed, and data can be processed on the fly. Furthermore the Astro-WISE system is very flexible and gives the astronomers powerful tools to analyse their data, using a common approach and a common data model. All of this has been connected to the EGEE Grid as one of the suppliers of storage and computing.
What differentiates the LOFAR information system from other Storage Grid environments is the key role for the metadata, the full data lineage, and the target-processing capability, where processing is started when necessary.
Conclusions and Future Work
The LOFAR Archive Information System is a key component of the LOFAR long-term archive, enabling researchers to find and process LOFAR data, while keeping track of the data lineage.
Future work will focus on further refinement of the processing and storage capabilities, using input obtained during processing and service challenges. To allow general use of the archive, without requiring end-user certificates, the use of robot certificates is planned for services that allow retrieving data and starting up processing. This also allows immediate coupling with the Virtual Observatory.
URL for further information | http://www.astro-wise.org/Lofar.html |
---|---|
Keywords | Astronomy, LOFAR, Data management, metadata, processing, Grid |