12-16 April 2010
Uppsala University
Europe/Stockholm timezone

Services for advanced workflow programming on gLite with WS-PGRADE portal

Apr 15, 2010, 9:40 AM
20m
Room IX (Uppsala University)

Room IX

Uppsala University

Oral Programming environments Workflow Management

Speaker

Gergely Sipos (MTA SZTAKI)

Description

After the great success of the P-GRADE Portal MTA SZTAKI has created the second generation P-GRADE portal, called WS-PGRADE. WS-PGRADE is a Web based environment for both developers of grid based dataflow applications, and end users who need transparent access to grid based application services. The presentation gives an overview of the application developer functionalities of the WS-PGRADE environment, details the various application patterns that can be realized in the system and executed on cluster based and PC based platforms.

Impact

Since WS-PGRADE can support both advanced application developers who understand grid technology as well as end-users who do not necessarily know grid systems, WS-PGRADE can attract large number of new users for EGEE. The built-in application repository concept enables the collaboration of user communities to jointly develop new applications. Application developers can place grid-enabled applications into the repository and end-users can download and use those applications by simply defining the necessary parameters. The applications can be single jobs, parameter sweep jobs, single workflows, parameterized workflows, workflows including parameterized nodes, data generators and data collectors. These special node types can be used anywhere in the workflow providing a very flexible way of constructing highly sophisticated and very large workflows. WS-PGRADE workflows can be executed in various resources: local clusters, Web Services, LCG-2, gLite, GT2, GT4 and BOINC grids, and hence users can exploit many resources from different grids.

Conclusions and Future Work

WS-PGRADE has been developed as part of the EU CancerGrid project where it was used to port and run three applications on joint gLite-BOINC-database platform: Descriptor Calculation, Model Building and Property Prediction. The first generic and public release of gUSE has been opened for public in November 2009. MTA SZTAKI already collaborates with several research teams who need support for advance dataflow applications on top of gLite middleware.

Detailed analysis

WS-PGRADE extends the workflow programming concept of P-GRADE Portal in many respects:
1. WS-PGRADE enables the development of nested workflows, recursive workflows, allow programmers to add control mechanism to the workflow graphs.
2. With the extended parameter study support data generator and data collector nodes can be placed anywhere in a workflow graph.
3. Components of a WS-PGRADE workflow can be Web services, jobs running on cluster grids (e.g. gLite, Globus, local cluster), tasks running on PC grids (e.g. BOINC), database queried performed on a JDBC compatible data repository.
4. WS-PGRADE supports the concept of abstract workflows, workflow templates and running workflow instances. These entities simplify the interaction among workflow developers, the collaboration of workflow developers with end users.
On top of that, the WS-PGRADE system is not a monolithic service any longer, it is a set of interoperable Web Services. Besides better scalability and fault tolerance, this approach also brings modularity and flexibility to WS-PGRADE programmers, who can from now replace different parts of the system with custom implementations.

URL for further information www.wspgrade.hu
Keywords Grid programming, workflow, environment, dataflow, SOA

Primary author

Gergely Sipos (MTA SZTAKI)

Presentation materials