12-16 April 2010
Uppsala University
Europe/Stockholm timezone

A novel approach to Workflow Management in Grid Environments

Apr 15, 2010, 9:00 AM
20m
Room IX (Uppsala University)

Room IX

Uppsala University

Oral Software services exploiting and/or extending grid middleware (gLite, ARC, UNICORE etc) Workflow Management

Speakers

Mr Frank Berretz (FH Aachen University of Applied Sciences) Mr Sascha Skorupa (FH Aachen University of Applied Sciences)

Description

Most workflow management systems in Grid environments provide a push-oriented job distribution strategy, where jobs are explicitly delegated to resources. In those scenarios dedicated resources execute jobs according to the request of a workflow engine or Grid wide scheduler. This approach has various limitations, particularly if human interactions should be integrated in workflows. To support such interactions with the benefit of enabling volunteering approaches, this presentation proposes a pull-based approach, where resources can actively select tasks for execution from a repository.

Detailed analysis

So far, workflow management systems in e-Science have focused on automatic job-execution. The actual schedule of a concrete workflow is basically a result of a decision process that relies on the workflow’s history and the availability of resources in VOs. Human interactions, either within the decision process or with respect to tasks in a workflow, are not considered yet or rather cumbersome to implement. The established push-oriented job distribution in Grids imposes execution patterns that also limit scalability due to central decision points and that do not provide flexibility in the above discussed sense. The presentation proposes a pull-based approach by implementing a task repository mediating between resources and concrete workflows. This concept is based on the emerging WS-HumanTask standard and allows resources to request jobs according to their individual workload and capabilities. We implemented a prototype by extending the existing jBPM engine, so that tasks are sent to an external repository. These tasks are accessible via a Java client using Web Service Interfaces. Further challenges consist of open security issues, UNICORE integration and the proof of scalability.

Conclusions and Future Work

The proposed solution is an evolutionary step towards the support of pull-based approaches in Grid environments resulting in better scalability and flexibility of current workflow executions. By integrating humans and special resources into workflows, this approach provides mentioned significant benefit. The presentation summarizes first results of the HiX4AGWS project that is gracefully supported by the BMBF. Several challenges still have to be solved. Particularly the prove of scalability, the UNICORE integration, and the implementation of a security infrastructure are work in progress.

Impact

Instead of selecting resources by schedulers, in a pull-based approach the autonomous resources decide about the execution of a particular job. SLAs between resources and VOs as well as market-based approaches are viewed as support technologies for steering this selection process efficiently. By avoiding a central brokering strategy performed by a single scheduler, improved scalability is feasible. Also, the traditional role of resources is rather passive, since the enactment is done by a central entity, leading to the requirement of stringent VO management, which has to be continuously transferred to the user mapping lists of site administrators. Among the high administration complexity, this approach is not transferable to a more flexible workflow solution that relies on human interactions. Pilot-based infrastructures already propose a pull-oriented approach to realise a late job binding. However, pilot-jobs are still bound to a push-based distribution strategy, even though they provide a vehicle for implementing a pull-oriented concept for batch-oriented jobs. But neither human interaction, e.g. required in e-Health, nor resource-driven contributions are appropriately supported.

Keywords workflow management, human interaction, pull-based, resource scheduling, UNICORE
URL for further information http://www.fh-aachen.de/17458.html

Primary authors

Mr Frank Berretz (FH Aachen University of Applied Sciences) Mr Sascha Skorupa (FH Aachen University of Applied Sciences)

Co-authors

Prof. Adam Belloum (University of Amsterdam) Dr Bernd Schuller (Research Center Jülich) Prof. Volker Sander (FH Aachen University of Applied Sciences)

Presentation materials