31 October 2022 to 3 November 2022
Clarion hotel Umeå
Europe/Amsterdam timezone

Lessons learned from an atlas data movement stress test for the Lancaster WLCG Tier-2's new XROOTD/CEPHFS storage.

3 Nov 2022, 14:05
25m
Clarion hotel Umeå

Clarion hotel Umeå

Storgatan 36, Umeå, Sweden
Storage & Filesystems Storage and Filesystems

Speaker

Matthew Steven Doidge (Lancaster University (GB))

Description

In an attempt to stress test and gauge the throughput potentialof the new XRootD fronted CEPHFS storage element at the Lancaster WLCG Tier 2 a large scale atlas transfer of 100TB of data was initiated. This volume of data was pushed to the site over a period of 3 days and revealed unexpected bottlenecks and problems - of which raw network bandwidth was not one of them. The most notable issue was the timely calculation of the transferred file's checksums. These results have changed the site design to horizontally expand our xrootd infrastructure in response. This talk will detail the tests, our observations and conclusions, and the site's plans after taking the results into account.

Desired slot length 10-15
Speaker release Yes
Presentation will be held... remotely

Primary author

Matthew Steven Doidge (Lancaster University (GB))

Co-authors

James William Walder (Science and Technology Facilities Council STFC (GB)) Duncan Rand

Presentation materials