EGEE'06

Climate data analysis on EGEE

26 Sept 2006, 17:00

2h 30m

CICG

Board: 41

Demo Users & Applications Demo session

Speakers

Dr Kerstin Ronneberger (DKRZ)Dr Stephan Kindermann (DKRZ)

Description

Climate research is generally very data-intensive. Observations, analysis and output data of climate simulations are traditionally stored in large archives and central databases. In order to make this data searchable and accessible for further analysis and/or data comparison on the grid appropriate interfaces between the existing data storage systems and the EGEE infrastructure need to be established. This interface is realized in close collaboration with the German C3-Grid project, which currently establishes a common metadata model, and metadata publishing infrastructure as well as data access mechanism. With respect to metadata these developments are based on international standards like the ISO 19115 / 19139 XML metadata schema and the OAI (open archives initiative) metadata harvesting approach. For data access a common community web service interface is developed which triggers complex data selection procedures and makes the result available on gridftp accessible storage. As a prerequisite for the data access a metadata catalogue, containing information for discovery and access of available datasets is needed. To make the metadata descriptions of the existing databases available on the EGEE infrastructure the AMGA catalogue is used. The catalogues is updated by means of XML descriptions, which are harvested on a regular basis from the existing datacenters. The upload of discovered data files from the external database to the EGEE infrastructure is based on webservices and gridftp: Based on information from the metadata catalogue the appropriate webservice endpoint is contacted from within an EGEE data analysis job. This webservice triggers data preprocessing functionality and delivers the result to a local gridftp accessible workspace from where they can be retrieved, stored on an SE and registered as EGEE accessible. In order to exemplify the added value for the climate community we will show components of a complete workflow of searching, accessing and analysing climate data supported by the established interfaces.