21–25 May 2012
New York City, NY, USA
US/Eastern timezone

AliEn Extreme JobBrokering

22 May 2012, 13:30
4h 45m
Rosenthal Pavilion (10th floor) (Kimmel Center)

Rosenthal Pavilion (10th floor)

Kimmel Center

Poster Distributed Processing and Analysis on Grids and Clouds (track 3) Poster Session

Speaker

Pablo Saiz (CERN)

Description

The AliEn workload management system is based on a central job queue wich holds all tasks that have to be executed. The job brokering model itself is based on pilot jobs: the system submits generic pilots to the compuiting centres batch gateways, and the assignment of a real job is done only when the pilot wakes up on the worker node. The model facilitates a flexible fair share user job distribution. This job model has proven stable and reliable over the past years and has surviced well the first two years of LHC operation with very little changes. Nonetheless there are several areas where the model can be pushed to the next level: most notably in the area of 'just in time' job and data assignment, where the decisions will be based on data closeness (relaxed locality) and the data which already has been processed. This methods will have significant efficiency enhancement effect for end user analysis tasks.

Summary

This contribution will describe the new mechanism implemented in AliEn for the job splitting and matchmaking. It will also show how the users
benefit from such a model.

Student? Enter 'yes'. See http://goo.gl/MVv53 no

Primary authors

Alina Gabriela Grigoras (CERN) Almudena Del Rocio Montiel Gonzalez (GSI - Gesellschaft fuer Schwerionen forschung (GSI)) Armenuhi Abramyan (Yerevan Physics Institute) Costin Grigoras (CERN) Dushyant Goyal (LNM Institute of Information Technology (IN)) Mr Federico Carminati (CERN) Jianlin Zhu (Central China Normal University (CN)) Latchezar Betev (CERN) Narine Manukyan (A.I. Alikhanyan National Scientific Laboratory (AM)) Pablo Saiz (CERN) Dr Stefano Bagnasco (I.N.F.N. TORINO) Steffen Schreiner (Technische Universitaet Darmstadt (DE)) Subho Sankar Banerjee (LNM Institute of Information Technology)

Presentation materials