Choose timezone
Your profile timezone:
CTA deployment meeting
→
Europe/Zurich
31/S-023 (CERN)
FTS:
- Eddy representing FTS today. Eddy to be added to castor-tape-dev.
- Explicit stager_rm from FTS: Full description in ticket. Julien will follow up with Andrea.
- Storage collocation hints:
- ATLAS proposes a pair of collocation hints to be transferred from Rucio to FTS. (Example: ['data15_13TeV.00284473.physics_Main.merge.AOD.f643_m1518','Main_AOD'], [<md5sum1>,'Main_AOD']).
- It is left up to each storage system to decide if and how to use them for data placement in order to optimise future reading. Initially, CTA will just receive the hints and log them.
- FTS does the translation into each back-end storage system (CTA, dCache, etc), similarly to what has been sketched for storage classes and hints.
- We propose a maximum string size of 64 chars per hint (so 128 chars in total)
- Andrea is in contact with ATLAS for agreeing on the metadata that Rucio will use for this purpose.
- Eric will propose and discuss with FTS team the format how to receive the metadata from FTS.
- ATLAS proposes a pair of collocation hints to be transferred from Rucio to FTS. (Example: ['data15_13TeV.00284473.physics_Main.merge.AOD.f643_m1518','Main_AOD'], [<md5sum1>,'Main_AOD']).
ATLAS Data Carousel recall exercise:
- Julien will present to ATLAS the outcome of the recall exercise (done after meeting, see link)
- GC problem (wrong space estimation causing too high cleanup) analysed and understood, fix being worked on
- Performance degraded also as a consequence of RAO only being executed on the first retrieve batch within a mount, which brings down retrieve performance during the rest of the mount (cf gitlab)
- When retrying failed file retrievals, these will be issuing a sparse tape access (few files remaining to be retrieved on each tape) which will result in lower performance.
EOS branching, tagging and merging
- Steve's proposal can be found attached to the agenda.
- EOS commits related to CTA go into a dedicated CTA branch based on a specific EOS (stable) tag. Untagged EOS commits are therefore left out. This requires EOS tags to be sufficiently stable.
Status of migration tools
- Tool for injecting fileID's to populate EOS has been developed by Andreas.
- File ID's will be the CASTOR nsfileids in both EOS and CTA catalogues.
- Michael's injection tools are still to be modified.
- What is the right hardware to run migration? Julien to check with Michael/Giuseppe.
Actions follow-up:
- Storage classes defined by ATLAS: atlas_raw, atlas_derived, atlas_default
- FIFO queueing is default
- Activities??
| who | what | by when |
| Eric | Agree with ATLAS on list of "activities" and configure via cta-admin. Deploy "activities" on ATLAS | 27/5 |
| Michael | Complete (with Cédric S.) namespace split-up | 30/5 |
| Cédric | Implement repacking disabled tapes once disabling is honored by CTA | 30/5 |
| Julien | Ensure CTA team is copied in exchanges with ATLAS and other experiments. | 24/5 |
| Julien | talk to procurement and network people (to ensure all network infrastructure is in place when nodes arrive) | 30/5 |
| Michael | Ensure that Georgios gets in touch with Luc to advance discussions on modelling collocation hints and assessing their usefulness. | 30/5 |
| Julien/Andrea | Explict stager_rm follow-up | 13/6 |
| Andrea | Agree Rucio->FTS metadata format for collocation hints and storage classes | 13/6 |
| Eric | propose and discuss with FTS team format how to receive collocation hints (in addition to storage classes and activities) from FTS. | 13/6 |
| Julien | Identify what is the right hardware to run migration | 13/6 |
There are minutes attached to this event.
Show them.
