Speaker
Mr
Ricky Egeland
(University of Minnesota – Twin Cities, Minneapolis, MN, USA)
Description
The CMS PhEDEx (Physics Experiment Data Export) project is responsible for facilitating large-scale data transfers across the grid ensuring transfer reliability, enforcing data placement policy, and accurately reporting results and performance statistics. The system has evolved considerably since its creation in 2004, and has been used daily by CMS since then. Currently CMS tracks over 2 PB of data in PhEDEx, and it has been tested well beyond the requirements of CMS.
Over the past year PhEDEx has evolved considerably, making use of new technologies (chiefly POE, an asynchronous, event-driven, cooperative-multitasking framework) and to consolidate the various components such that it is easy to reuse existing techniques and components in new features. This has resulted in changes to nearly every piece of the PhEDEx code base, creating a flexible modular framework. We are able to evolve the implementation to match changes in the requirements of the experiment, without changing the fundamental design.
Two major new features have recently been added to the PhEDEx system; an extensible data service and an improved transfer back-end module. The extensible data service provides machine-readable data over HTTP as the primary means of integration with other CMS services. An authenticated command line interface is also provided, making it possible to provide new utilities quickly with minimal development effort. The new transfer back-end module now integrates closely with FTS, the glite provided transfer tool, to provide accurate status information while keeping as much data in flight as possible. The new transfer back-end is transfer technology independent, and we expect to be able to support new transfer tools as they become available.
We describe the CMS PhEDEx system that is in place for CMS "first data taking" in 2008, provide details on the benefits and implementations of the new features, and describe other new tools that are now available.
Primary author
Mr
Ricky Egeland
(University of Minnesota – Twin Cities, Minneapolis, MN, USA)
Co-authors
Dr
Simon Metson
(Bristol University, Bristol, UK)
Dr
Tony Wildish
(Princeton University)
