21–25 Aug 2017
University of Washington, Seattle
US/Pacific timezone

Testing the limits of an LVS - GridFTP Cluster as a replacement for Bestman

21 Aug 2017, 16:50
20m
Auditorium (Alder Hall)

Auditorium

Alder Hall

Oral Track 1: Computing Technology for Physics Research Track 1: Computing Technology for Physics Research

Speaker

Edgar Fajardo Hernandez (Univ. of California San Diego (US))

Description

The Worldwide LHC Computing Grid (WLCG) is the largest grid computing infrastructure in the world pooling the resources of 170 computing centers (sites). One of the advantages of grid computing is that multiple copies of data can be stored at different sites allowing user access that is independent of that site's geographic location, unique operating systems, and software. Each site is able to communicate using software stacks collectively referred to as “middleware". One of the middleware pieces is the storage element (SE) which manages data access between sites.

The middleware distributed by the Open Science Grid (OSG) previously used a storage resource manager (SRM) allowing for sites to expose their SEs for access by off-site compute elements (CEs) via the Grid File Transfer Protocol (GridFTP). OSG is eliminating the use of an SRM entirely and transitioning towards a solution based solely on GridFTP and Linux Virtual Server (LVS). LVS is a core component of the Linux kernel, so this change increases both maintainability and interoperability. In this document, we outline our methodologies and results from the large scale testing of an LVS+GridFTP cluster for data reads. Additionally, we discuss potential optimizations to the cluster to maximize total throughput.

Primary authors

Brian Paul Bockelman (University of Nebraska-Lincoln (US)) Clifton Pottberg (University of California San Diego) Edgar Fajardo Hernandez (Univ. of California San Diego (US)) Terrence Martin (UCSD) Garhan Attebury (University of Nebraska-Lincoln) Frank Wuerthwein (UCSD)

Presentation materials

Peer reviewing

Paper