2–6 Mar 2009
Le Ciminiere, Catania, Sicily, Italy
Europe/Rome timezone

Debugging Data Transfers between CMS Computing Centres

2 Mar 2009, 17:30
25m
Raffaello (80) (Le Ciminiere, Catania, Sicily, Italy)

Raffaello (80)

Le Ciminiere, Catania, Sicily, Italy

Viale Africa 95100 Catania
Oral Grid Services exploiting and extending gLite middleware Data Management

Speaker

Dr Nicolo Magini (CERN IT-GS-EIS & INFN-CNAF)

Description

The Debugging Data Transfers (DDT) Task Force was created to coordinate the debugging of data transfer links among WLCG sites supporting the CMS Virtual Organization. The task force aimed to commission the most crucial transfer routes among CMS sites by designing and enforcing a clear procedure to debug problematic links. The preparation, activities and experience of the DDT Task Force are discussed. Common technical problems and challenges encountered are explained and summarized.

Impact

As of November 2008, a total of 474 links have achieved commissioning according to the metrics required by the DDT task force, which are currently:
- 20 MB/s averaged over 24 hours for links with source at the CERN Tier 0 or at a Tier 1
- 5 MB/s averaged over 24 hours for links with source at a Tier 2

The 474 commissioned links are divided as follows:
- All of the 56 links between CERN and the 7 T1s supporting the CMS VO are commissioned.
- 272 downlinks from T1s to T2s are commissioned, representing 77% of the total mesh of 352 links between T1s and T2s supporting the CMS VO which have joined the LoadTest infrastructure.
- 41 uplinks from T2s to the respective associated T1s are commissioned, representing 85% of the total. In addition, 88 uplinks between T2s and non-associated T1s are also commissioned.
- 17 T2-T2 cross-links are also commissioned.

The experience gained in link commissioning has been collected to provide a useful knowledge base for the sites in debugging transfer issues.

Keywords

HEP, Data Management

Conclusions and Future Work

A focused effort to debug transfer links within CMS sites proved to be useful in helping to maintain a working
system for data transfers, documenting common problems and solutions, and alerting site administrators to current problems. This effort continued in 2008 with requirements and testing exercises that more closely match the CMS computing model and expected data transfer patterns when data taking will resume in 2009.

URL for further information

https://twiki.cern.ch/twiki/bin/view/CMS/DDT

Detailed analysis

The CMS experiment at CERN is preparing for LHC data taking in several computing preparation activities. In 2007 a traffic load generator infrastructure was designed and deployed to equip the WLCG sites which support the CMS Virtual Organization with a means for debugging, load-testing and commissioning data transfer routes among CMS Computing Centres. The LoadTest is based upon PhEDEx as a reliable, scalable dataset replication system, which fully supports the gLite File Transfer Service as a backend for file replication. The Debugging Data Transfers (DDT) Task Force was created to coordinate the debugging of data transfer links by designing and enforcing a clear procedure to debug problematic links and by assisting the sites in solving technical problems. Links are tested against a set of agreed metrics in an independent debugging environment and moved to a production environment when the commissioning conditions are achieved. Common technical issues are documented for the sites.

Authors

James Letts (UCSD, La Jolla, CA USA) Jorgen D’Hondt (Vrije Universiteit Brussel (VUB), Belgium) Joris Maes (Vrije Universiteit Brussel (VUB), Belgium) Jukka Klem (Helsinki Institute of Physics, Finland) Dr Nicolo Magini (CERN IT-GS-EIS & INFN-CNAF) Petra Van Mulders (Vrije Universiteit Brussel (VUB), Belgium) Sherif Elgammal (Universite Libre de Bruxelles (ULB), Belgium)

Presentation materials