11-14 February 2008
<a href="http://www.polydome.org">Le Polydôme</a>, Clermont-Ferrand, FRANCE
Europe/Zurich timezone

The gLite Workload Management System

13 Feb 2008, 11:10
25m
Champagne (<a href="http://www.polydome.org">Le Polydôme</a>, Clermont-Ferrand, FRANCE)

Champagne

<a href="http://www.polydome.org">Le Polydôme</a>, Clermont-Ferrand, FRANCE

Oral Existing or Prospective Grid Services Workflow and Parallelism

Speaker

Francesco Giacomini (Istituto Nazionale di Fisica Nucleare (INFN))

Description

The WMS accepts requests concerning the execution of a computation, whose description is expressed in a flexible language, based on Condor ClassAds, as a set of key - value pairs. The WMS is then responsible to translate the logical description to concrete operations, in order to bring the execution of a job to a successful end. Several types of jobs are supported: simple, intra-cluster MPI, interactive, collections, parametrics, simple workflows in the form of DAGs, with on-going development of a generic workflow engine. Additional benefits concern sandbox management, with support for multiple transfer protocols, data-driven match-making, the availability of multiple mechanisms of error prevention and management, isolation from the heterogeneity of the infrastructure, the capability to implement optimizations based on non-local information, such as bulk submission and match-making. The progress of the job is safely recorded into the complementary Logging and Bookkeeping Service.

3. Impact

Managing a job, from submission to completion, usually involves the interaction with several other services: computing elements, storage elements, information systems, data catalogs, authorization, policy and accounting frameworks, credential renewal. Unfortunately their convergence towards standard solutions has not shown fast progress in the past, with the consequence that multiple implementations with different interfaces are available on the same infrastructure. The complexity that stems from this situation is also a major cause of errors. An important goal of the WMS is then to hide as much as possible to end users both the heterogeneity of infrastructure components and the occurence of non-fatal errors, without sacrificing generality and performance during request processing.

In order to ease the integration with higher-level middleware and application frameworks, the WMS itself exposes a Web Service interface compliant with the WS-I specification.

URL for further information:

http://egee-jra1-wm.mi.infn.it/egee-jra1-wm/

1. Short overview

The gLite Workload Management System has been designed and developed with the ambition to represent the main access point to the computing resources made available on a Grid. The goal is to provide a reliable, effective and efficient service responsible for the distribution and management of computational jobs, hiding the intrinsic complexity of the infrastructure to its users. The abstraction provided by the WMS is generic enough to support applications coming from largely different domains.

Provide a set of generic keywords that define your contribution (e.g. Data Management, Workflows, High Energy Physics)

Workload Management, interoperability, integration, abstraction, error management

4. Conclusions / Future plans

The WMS has been deployed in a number of different multi-user and multi-VO scenarios, thanks to the neutrality of its design. The recent introduction of features like the bulk match-making has shown that it can cope with sustained high loads. When used in demanding production environments it has nevertheless shown some limits in terms of stability and usability. With the experience gained in the past years, parts of the WMS are now being revised in order to fully comply with the expectations.

Primary author

Francesco Giacomini (Istituto Nazionale di Fisica Nucleare (INFN))

Co-authors

Alessandro Maraschini (Elsag Datamat) Alessio Gianelle (Istituto Nazionale di Fisica Nucleare (INFN)) Andrea Guarise (Istituto Nazionale di Fisica Nucleare (INFN)) Antonia Ghiselli (Istituto Nazionale di Fisica Nucleare (INFN)) Laura Perini (Istituto Nazionale di Fisica Nucleare (INFN)) Marco Cecchi (Istituto Nazionale di Fisica Nucleare (INFN)) Massimo Sgaravatto (Istituto Nazionale di Fisica Nucleare (INFN)) Salvatore Monforte (Istituto Nazionale di Fisica Nucleare (INFN))

Presentation Materials