Discuss LHCb use-case in terms of non-DIRAC data archival and present potential solutions.
This data is NOT physics data, but detector-related technical data. Therefore, this data does not fit in the DIRAC workflow.
Two different data location were identified:
Histogram and tuples data stored in Point 8 Netapp flash drive: 11PB for ~7.5 million files
Detector calibration data stored in /eos/lhcb/hlt2_save/ : 1.1PB initial + between 100TB and 500TB. This data is created per RUN and has a lifetime of the RUN.
This data will be rarely accessed by around 10-20 people and therefore should be kept on tape.
independently of the solution to the above use-case, it has to be exposed via the IT engagement channels to be endorsed as a new workflow for LHCb and use-case clearly scoped.
The EOS-archive tool current usage was presented in a presentation https://cernbox.cern.ch/s/SQKUkYRFP6C4jIf
It was made clear that this tool has to be used for non-critical use-cases and will provide archival and retrieval functionality on a best-effort basis.
No commitment was made on using this solution. It was made clear that this proposal had to go to the different IT engagement channel in order to make this new activity visible and that resources are allocated.
After some discussions, it was noticed that the LHCb use-case could fit into the eosctapublicdisk instance where the architecture is presented as follow:
Some transfer/archival tests will be performed between F. HEMMER and J. LEDUC.