3rd EGEE User Forum

Solving Data Transfer Level Grid Interoperability among SRM, SRB and OGSA-DAI data resources

12 Feb 2008, 16:00

1m

Exhibition Hall (<a href="http://www.polydome.org">Le Polydôme</a>, Clermont-Ferrand, FRANCE)

Demonstration Existing or Prospective Grid Services Demonstrations

Speaker

Mr Gergely Sipos (MTA SZTAKI)

Description

The input files of a node (job or service call) of a P-GRADE workflow can come from file systems, like SRM or SRB, or from database management systems via OGSA-DAI and the results can also be fed into any of these solutions. Both the file systems and the databases can be located in different production grids, and the jobs of the workflow can also be mapped to different grids. These grids can be based on different grid middleware and may require different user certificates for authentication. The workflow level data resource integration allows the seamless interoperation of SRB catalogues, GridFTP file systems and EGEE storage elements (based on SRM) at the level of P-GRADE workflows. We will demonstrate that jobs of an urban traffic simulation workflow are running in different grids (US OSG, UK NGS, EGEE) and utilise data resources based on different technologies (SRB, SRM, GridFTP, local) from these different grids.

If demonstration is requested please explain what visual or interactive aspects of the contribution necessitate a demonstration rather than a presentation or poster?

The rich set of data access possibilities and grid interoperation features of P-GRADE portal necessitates a demonstration. A short presentation is not enough to show the interested potential users all the possible usage scenarios. Demonstration gives better understanding of the grid interoperation concepts both at the job submission and at the data resource access level. We will use several real applications (urban traffic simulation, e-market place, etc.) during the demonstration.

Provide a set of generic keywords that define your contribution (e.g. Data Management, Workflows, High Energy Physics)

Grid workflow, interoperation, grid data resources, SRB, OGSA-DAI

1. Short overview

The GIN VO of OGF defined 4 levels of solving interoperation among different grids. It was demonstrated several times how P-GRADE portal can support job submission level interoperation. In the current demonstration we show how P-GRADE portal was extended with SRM, SRB and OGSA-DAI portlet support in order to solve the interoperation problem at the data movement level. Moreover, interoperation of SRM, SRB and OGSA-DAI data resources are supported among nodes of a workflow.

3. Impact

Grid portals typically do not provide SRM, SRB and OGSA-DAI portlets or they provide only one of them in a limited form. We have developed intelligent versions of these portlets and showed how to integrate them into a single portal (P-GRADE). As a result users of P-GRADE portal can easily access all the major grid-related file and database systems without learning the different command line interfaces.
Moreover, grid workflow systems are typically tailored to one particular grid concerning both job submission and data file access mechanisms. P-GRADE portal is the first multi-grid portal where not only the job submission is supported among different grids but also the data resource access mechanisms of different grids can become interoperable at workflow level. Intra-workflow interoperation of grid data resources allows data to be input from or output to different file storage systems or database solutions, located in several different grids.

4. Conclusions / Future plans

There are several variants of P-GRADE portal deployed for different user communities. For SEE-GRID and EGEE VOs P-GRADE portal is deployed with GridFTP file systems and EGEE storage elements (based on SRM). For the UK NGS P-GRADE portal is deployed with full access to GridFTP, SRB and SRM file systems. Another experimental version of the portal contains additionally the OGSA-DAI portlet. A new version of the portal that integrates all these features is planed for the first half of 2008.