Speaker
Shaun de Witt
(STFC)
Description
For many years the Storage Resource Manager (SRM) has been the de-facto federation technology used by WLCG. This technology has, along with the rest of the middleware stack, mediated the transfer of many Petabytes of data since the start of data taking. In recent years, other technologies have become more popular as federation technologies because they offer additional functionalities that are not provided by SRM or they are based on more widely adopted standards. The main technologies currently used or being evaluated with WLCG are xrootd and WebDav.
In this work we investigate the overhead associated with each technology for different commonly used storage systems within WLCG. For jobs running at a local site, delays in file access can make a significant contribution to job efficiency and it is this we report on results from both controlled tests and in 'real life' operations. While it is difficult to compare the overheads across different installations and hardware configurations, a comparison within an installation can help inform experiments to make the best choice of technology at any site and for any use case.
Primary author
Shaun de Witt
(STFC)
Co-authors
Paul Millar
(Deutsches Elektronen-Synchrotron (DE))
Wahid Bhimji
(University of Edinburgh (GB))
