Speaker
Wei Yang
(SLAC National Accelerator Laboratory (US))
Description
Rubin/LSST is using the Rucio/FTS/Xrootd/dCache ecosystem for data transfer and distribution between their Fr/UK/US data facilities and independent data access centers around the world. Here we will outline the integration of Rubin's native data management system Butler and the Rucio ecosystem. We will focus on Rubin's specific challenge of efficiently transferring large number of small files, and will discuss the lesson we learned and tuning we did during recent large scale transfers and distributions.
Author
Wei Yang
(SLAC National Accelerator Laboratory (US))