Speaker
Description
The Dynafed data federator is designed to present a dynamic and unified view of a distributed file repository. We describe our use of Dynafed to construct a production-ready WLCG storage element (SE) using existing Grid storage endpoints as well as object storage. In particular, Dynafed is used as the primary SE for the Canadian distributed computing cloud systems. Specifically, we have been using a Dynafed-based SE in production for reading input files of Belle-II jobs. We have run up to 6000 Belle-II jobs simultaneously on distributed cloud resources, requiring the transfer of approximately 60 TB of input data per day. Similarly, we have been using a Dynafed-based SE in pre-production testing for the ATLAS experiment. We will describe the configuration of Dynafed to make it suitable as a WLCG SE. In particular, we will highlight the improvements within Dynafed that make it possible to do checksum based file verification and 3rd-party file copy via WebDAV. We will also report on a new monitoring system and an automated system that collects storage information from all endpoints. The monitoring is accessible via a web browser and also on the command line. It includes, for example, information of who requested files, which endpoints served the files, how often each file was requested from an endpoint, as well as endpoint storage information collected by the system we developed. The storage information system collects information about each storage endpoint which is then used by Dynafed to identify endpoints with sufficient storage capacity. The presentation will discuss the configuration and successful operation of Dynafed within the context of a distributed compute cloud infrastructure.
| Consider for promotion | Yes |
|---|
