A Comparison of the Overheads Associated with WLCG Federation Technologies

Not scheduled
15m
OIST

OIST

1919-1 Tancha, Onna-son, Kunigami-gun Okinawa, Japan 904-0495
poster presentation Track3: Data store and access

Speaker

Shaun de Witt (STFC)

Description

For many years the Storage Resource Manager (SRM) has been the de-facto federation technology used by WLCG. This technology has, along with the rest of the middleware stack, mediated the transfer of many Petabytes of data since the start of data taking. In recent years, other technologies have become more popular as federation technologies because they offer additional functionalities that are not provided by SRM or they are based on more widely adopted standards. The main technologies currently used or being evaluated with WLCG are xrootd and WebDav. In this work we investigate the overhead associated with each technology for different commonly used storage systems within WLCG. For jobs running at a local site, delays in file access can make a significant contribution to job efficiency and it is this we report on results from both controlled tests and in 'real life' operations. While it is difficult to compare the overheads across different installations and hardware configurations, a comparison within an installation can help inform experiments to make the best choice of technology at any site and for any use case.

Primary author

Co-authors

Paul Millar (Deutsches Elektronen-Synchrotron (DE)) Wahid Bhimji (University of Edinburgh (GB))

Presentation Materials

There are no materials yet.