Speaker
Description
Impact
We expect about 350GB of data coming from the telescope each night of observation so it would be likely to ingest, process and store this volume within 24 hours.
To reach the 24 hours time constraint, each science image reduction will be distributed to an independent job, so multiple cores will run in parallel at this stage. Such a technical challenge will be only possible into the grid framework.
The raw images and the final output catalogues are planned to be stored in tapes. The grid environment will make them available to all the VO members who, from their own institutions will be able to perform data analysis and cosmological studies using the unique quantity and quality of data that the PAU survey will provide.
The experience of PIC to deal with astronomical data, coming from the support it is giving to the MAGIC experiment, will be taken as a starting point and further optimized, for advantage of the two scientific groups.
Conclusions and Future Work
The PAU survey will provide data of a huge number of galaxies that will be catalogued to form a new map of the universe in large scale. A high volume of data will be managed, processed and stored using the facilities of the PIC computing center and the grid framework. The data are intended to give new information for the understanding of the physics of the accelerated universe and the dark energy.
Detailed analysis
The PAU survey camera will scan a large area in the sky with an extensive set of filters. The data consists of astronomical images in FITS format. The process of obtaining the calibrated galaxy catalogues from the raw astronomical images is called data reduction. Simulated images are used as input for the validation of the process, since in the simulation the exact properties from the input galaxies and stars are known. The matching software used for validation is written in ROOT framework.
The reduction pipeline, written in python, consists of initializing the system, producing master calibration images and obtaining the catalogues from the science images.
As the code needs to access to a large number of files during the execution, the raw images, permanently stored in tapes, are temporally copied in a nfs disk, providing a significantly quicker and more efficient access.
A grid environment is the optimal location to carry out these computationally intense and loaded tasks.
| URL for further information | http://www.ice.csic.es/research/PAU/PAU-welcome.html |
|---|---|
| Keywords | Cosmology and Astrophysics, Data Management |
