CTA deployment meeting

Thursday 2 Apr 2020, 14:00 → 16:00 Europe/Zurich

600/R-001 (CERN)

Michael Davis (CERN)

Welcome

• Paul Vriet was introduced to the team. Paul will be working on RAO for LTO drives.

EOSCTAATLAS Testing/Putting EOSCTAATLAS into Production

• Julien reported in detail on the results of the tests (See CTA Testing: ATLAS).
• The recall tests and SFO tests were successful. The only outstanding issue is that SFO are repeatedly scanning the EOS namespace to work out the filename for a small proportion of transferred files. Julien proposed several solutions which were later discussed at the IT-ST/ATLAS Storage Meeting and IT/ATLAS Coordination Meeting on 6 April.
• Michael presented a summary of the test results at the IT/ATLAS Coordination Meeting. On the issue of the namespace scanning, the "FTS checks whether file is on tape" solution was preferred by ATLAS. This requires implementation of a feature in FTS which is expected to be done by the summer.
• Julien is continuing with ATLAS simultaneous read/write stress test.

Conclusions

• There are no blocking issues to go to production. We continue with the plan, albeit on a more conservative timeline due to communication difficulties and delays from Covid-19 situation.
• In order not to lose time, we will not wait for ATLAS before we start commissioning the other LHC experiments. This can be done in parallel.

EOSCTACMS Testing

• CMS Rucio + FTS + EOS + CTA tests started this week.
• Julien is helping with the FTS multi-hop fix.

EOSCTAALICE Testing

• ALICE are planning a recall campaign starting end of April and continuing until July.
• We would like to take this opportunity to use CTA instead of CASTOR for this campaign
• Lack of backpressure/Garbage Collector is not a blocker for this campaign as it is envisaged that there will be sufficient spinner disk space to recall all the files (~3 PB).
• This will be proposed to ALICE at the meeting on 8 April.
• In the meantime, we will continue with setting up an instance to test the new Garbage Collector. It remains to be seen if the GC test and recall exercise will be done on the same test instance or two separate test instances.

Other Issues for Going to Production

• ATLAS repack should be finished by end of April.
• We would like to develop a tool to import tapes written in CTA into CASTOR, to give us a way back in a disaster recovery scenario. Michael could write the part to scan the EOS namespace and CTA catalogue and extract all the data for a given tape to a temporary table. It would be better if the CASTOR injection side could be done by a CASTOR expert...
• Julien will order a CERN phone.   

AOB

• As we are parallelising our activities, Michael will reorganise the open tickets from the current "milestone-based" way to a more Kanban dashboard way so that all the parallel activities can be seen and prioritised together.
• CASTOR issues: Oliver will set up a rota to cover for Eric while he is on leave.

 

14:00 → 14:20 Putting EOSCTAATLAS into production

• Tool to import tapes written in CTA into CASTOR.
• Asking for support from EOS operations team for CTA disk cache.
• CERN phone for Julien?
• Final commissioning tests.
• Final configuration of the EOSCTATLAS instance.
• ATLAS repack finished by end of April.
• Presentation at IT/ATLAS coordination meeting on Monday to include: 1. Results of reprocessing campaign / SFO tests. 2. Are ATLAS satisfied with the results of the SFO tests? 3. Proposed date for going to production.

14:20 → 14:40 EOSCTAALICE Testing

• Here is the discussion document on the EOSCTAALICE instance configuration
• We don't have a way to do tape → SSDs → spinners for recalls. Julien points out that this means that backpressure will not work.
• Discuss the backpressure problem and how to set up the spinner space so we can start testing with Steve's GC.

14:40 → 14:45 AOB