Jul 9 – 13, 2018
Sofia, Bulgaria
Europe/Sofia timezone

Bootstrapping a New LHC Data Transfer Ecosystem

Jul 11, 2018, 12:30 PM
Hall 8 (National Palace of Culture)

Hall 8

National Palace of Culture

presentation Track 4 - Data Handling T4 - Data handling


Brian Paul Bockelman (University of Nebraska Lincoln (US))


GridFTP transfers and the corresponding Grid Security Infrastructure (GSI)-based authentication and authorization system have been data transfer pillars of the Worldwide LHC Computing Grid (WLCG) for more than a decade. However, in 2017, the end of support for the Globus Toolkit - the reference platform for these technologies - was announced. This has reinvigorated and expanded efforts to replace these pillars. We present an end-to-end alternate utilizing HTTP-based WebDAV as the transfer protocol, and bearer tokens for distributed authorization.

This alternate ecosystem, integrating significant pre-existing work and ideas in the area, adheres to common industry standards to the fullest extent possible, with minimal agreed-upon extensions or common interpretations of the core protocols. The bearer token approach allows resource providers to delegate authorization decisions to the LHC experiments for experiment-dedicated storage areas.

This demonstration touches the entirety of the stack - from multiple storage element implementations to FTS3 to the Rucio data management system. We show how the traditional production and user workflows can be reworked utilizing bearer tokens, eliminating the need for GSI proxy certificates for storage interactions.

Primary authors

Brian Paul Bockelman (University of Nebraska Lincoln (US)) Andrew Bohdan Hanushevsky (SLAC National Accelerator Laboratory (US)) Oliver Keeble (CERN) Mario Lassnig (CERN) Paul Millar (DESY) Derek John Weitzel (University of Nebraska-Lincoln (US)) Wei Yang (SLAC National Accelerator Laboratory (US))

Presentation materials