Speaker
David Cameron
(University of Oslo (NO))
Description
Staging data to and from remote storage services on the Grid for users' jobs is a vital component of the ARC computing element. A new data staging framework for the computing element has recently been developed to address issues with the present framework, which has essentially remained unchanged since its original implementation 10
years ago. This new framework consists of an intelligent data transfer scheduler which handles priorities and fair-share, a rapid caching system, and the ability to delegate data transfer over multiple nodes to increase network throughput. This paper uses data from real user
jobs running on production ARC sites to present an evaluation of the new framework. It is shown to make more efficient use of the available
resources, reduce the overall time to run jobs, and avoid the problems seen with the previous simplistic scheduling system. In addition, its
simple design coupled with intelligent logic provides greatly increased flexibility for site administrators, end users and future development.
Authors
Mr
Aleksandr Konstantinov
(VILNIUS UNIVERSITY)
Andrej Filipcic
(Jozef Stefan Institute (SI))
David Cameron
(University of Oslo (NO))
Dmytro Karpenko
(University of Oslo)