3-7 November 2008
Ettore Majorana Foundation and Centre for Scientific Culture
Europe/Zurich timezone

Using constraint programing to resolve the multi-source / multi-site data movement paradigm on the Grid

4 Nov 2008, 16:35
25m
Ettore Majorana Foundation and Centre for Scientific Culture

Ettore Majorana Foundation and Centre for Scientific Culture

Via Guarnotta, 26 - 91016 ERICE (Sicily) - Italy Tel: +39-0923-869133 Fax: +39-0923-869226 E-mail: hq@ccsem.infn.it
Parallel Talk 1. Computing Technology Computing Technology for Physics Research - Session 2

Speaker

Mr. Michal ZEROLA (Nuclear Physics Inst., Academy of Sciences, Praha)

Description

In order to achieve both fast and coordinated data transfer to collaborative sites as well as to create a distribution of data over multiple sites, efficient data movement is one of the most essential aspects in distributed environment. With such capabilities at hand, truly distributed task scheduling with minimal latencies would be reachable by internationally distributed collaborations (such as ones in HENP) seeking for scavenging or maximizing on geographically spread computational resources. But it is often not all clear (a) how to move data when available from multiple sources or (b) how to move data to multiple compute resources to achieve an optimal usage of available resources. Constraint programming (CP) is a technique from artificial intelligence and operations research allowing to find solutions in a multi-dimensional space of variables. We present a method of creating a CP model consisting of sites, links and their attributes such as bandwidth for grid network data transfer also considering user tasks as part of the objective function for an optimal solution. We will explore and explain trade-off between schedule generation time and divergence from the optimal solution and show how to improve and render viable the solution's finding time by using search tree time limit, approximations, restrictions such as symmetry breaking or grouping similar tasks together, or generating sequence of optimal schedules by splitting the input problem. Results of data transfer simulation for each case will also include a well known Peer-2-Peer model, and time taken to generate a schedule as well as time needed for a schedule execution will be compared to a CP optimal solution. We will additionally present a possible implementation aimed to bring a distributed datasets (multiple sources) to a given site in a minimal time.

Primary authors

Dr. Jerome LAURET (BROOKHAVEN NATIONAL LABORATORY) Mr. Michal ZEROLA (Nuclear Physics Inst., Academy of Sciences, Praha)

Co-authors

Dr. Michal SUMBERA (Nuclear Physics Inst., Academy of Sciences, Praha) Dr. Roman BARTAK (Faculty of Mathematics and Physics, Charles University, Praha)

Presentation Materials

Your browser is out of date!

Update your browser to view this website correctly. Update my browser now

×