Speaker
T. Barrass
(CMS, UNIVERSITY OF BRISTOL)
Description
CMS currently uses a number of tools to transfer data which, taken together, form
the basis of a heterogenous datagrid. The range of tools used, and the directed,
rather than optimised nature of CMS recent large scale data challenge required the
creation of a simple infrastructure that allowed a range of tools to operate in a
complementary way.
The system created comprises a hierarchy of simple processes (named agents) that
propagate files through a number of transfer states. File locations and some
application metadata were stored in POOL file catalogues, with LCG LRC or MySQL
backends. Agents were assigned limited responsibilities, and were restricted to
communicating state in a well-defined, indirect fashion through a central transfer
management database. In this way, the task of distributing data was easily divided
between different groups for implementation.
The prototype system was developed rapidly, and achieved the required sustained
transfer rate of ~10 MBps, with O(10^6) files distributed to 6 sites from CERN.
Experience with the system during the data challenge raised issues with underlying
technology (MSS write/read, stability of the LRC, maintenance of file catalogues,
synchronisation of filespaces _) which have been successfully identified and
handled. The development of this prototype infrastructure allows us to plan the
evolution of backbone CMS data distribution from a simple hierarchy to a more
autonomous, scalable model drawing on emerging agent and grid technology.
Primary authors
A. Afaq
(FNAL)
A. Fanfani
(INFN-Bologna)
C. Charlot
(IN2P3, LLR)
C. Grandi
(INFN-Bologna)
D. Bonacorsi
(CNAF-INFN)
D. Newbold
(University of Bristol)
I. Fisk
(FNAL)
J. Hernandez
(CIEMAT Madrid)
J. Rehn
(University of Karlsruhe)
K. Rabbertz
(University of Karlsruhe)
L. Tuura
(Northeastern University)
M. Ernst
(FNAL)
O. Maroney
(University of Bristol)
P. GARCIA-ABIA
(CIEMAT)
S. Metson
(University of Bristol)
T. Barrass
(CMS, UNIVERSITY OF BRISTOL)
T. Wildish
(Princeton)
W. Jank
(CERN)
Y. Wu
(FNAL)
