Description
Belle II experiment can take advantage from Data federation technologies to simplify access to distributed datasets and file replicas. The increasing adoption of http and webdav protocol by sites, enable to create lightweight solutions to give an aggregate view of the distributed storage.
In this work, we make a study on the possible usage of the software Dynafed developed by CERN for the creation of an on-the-fly data federation.
We created a first dynafed server, hosted in the datacentre in Napoli, and connected with about the 50% of the production storages of Belle II. Then we aggregated all the file systems under a unique http path. We implemented as well an additional view, in order to browse the single storage file system.
On this infrastructure, we performed a stress test in order to evaluate the impact of federation overall performances, the service resilience, and to study the capability of redirect clients properly to the file replica in case of fault, temporary unavailability of a server.
The results show a good potentiality of the service and suggest additional investigation for additional setup.
| Primary Keyword (Mandatory) | Data processing workflows and frameworks/pipelines |
|---|---|
| Secondary Keyword (Optional) | Distributed data handling |
| Tertiary Keyword (Optional) | Storage systems |
