CTA deployment meeting

Friday 25 Jun 2021, 14:00 → 15:30 Europe/Zurich

600/R-001 (CERN)

Michael Davis (CERN)

Cold Data

Data will be migrated after the big summer repack campaign.

 

User Data

Next week we will announce to CASTOR users that CASTOR will be shut down on 31 January 2021. Users can be given a migration path to CERNBox. Around 20 active users can be contacted one-by-one. Anything left over at the end of this campaign will be put into cold storage in CTA.

 

Backup

We will split Hadoop into a separate use case, and split backups into "archive files and never delete them" vs. "rotate files and delete old files" tapepools. The first case can be migrated sooner, the second case needs repack to be working in CTA so will be done later.

14:00 → 14:10 Recent migrations

CLIC/ILC

• #310 CLIC/ILC tests on EOSCTAPUBLIC PPS
• #392 Migrate CLIC/ILC from CASTOR to CTA
• Everything went smoothly, no sign of the NULL column glitch

14:10 → 14:20 Upcoming migrations

Tuesday 29 June: n_TOF

• #143 nTOF tests on EOSCTAPUBLIC PPS , #161 Spinners space on EOSCTA PPS
• #327 Migrate n_ToF from CASTOR to CTA

Wednesday 30 June: AMS

• #82 AMS tests on EOSCTAPUBLIC PPS
• #305 Migrate AMS data from CASTOR to CTA

14:20 → 14:30 CTA PUBLIC Migration from CASTOR

• Migration of SMEs from CASTOR to CTA, see Putting EOSCTAPUBLIC into Production
• Schedule for migrating CASTOR PUBLIC experiments to CTA

Cold Data

• NA48 - #341 - repack underway
• r_public_misc: covers most of the rest of the small experiment use cases (single copy). Repacking will separate from r_public_user.
• r_public_user_[1,2] is other small experiment data (not user data despite the name of the tapepool), dual copy. Merged with r_lep_data_[1,2].

User Data

• #390 Migration of CASTOR user accounts to CTA
• Experiment data has been separated from user data as far as possible
• New do_not_migrate fileclass for data which should probably be deleted
• Separation of Service Accounts, Secondary Accounts and Primary Accounts
• Migration of active users could be to CERNBox rather than CTA, we could start a campaign in the autumn
• Only around 20 active users of CASTOR home directories (who wrote files in the last year). Almost all of these are experiment analysis or MC use cases which could possibly be migrated to experiment areas of CTA.

Ready for Big Summer Repack.

Backup

• #307 Migrate CASTOR Backup use cases to CTA
• /afsmigration #282.

Use cases:

1. EOS archive. Physics data archived from EOS using Elvin's tool. Encryption not needed. r_archive_[1,2].
2. Project archival. Write once, keep forever. No rotation of data, archive just keeps getting bigger. May include private data? Needs encryption. /hdfsbackup, /afsmigration
3. "Normal" backup with deltas, rotation, encryption.

Use cases (2) and (3) could be separated and migrated separately to CTA.

14:30 → 14:40 CTA PUBLIC New Use Cases

• Vlado discussed use cases with EOS team. The EOS archive tool seems to match the needs of these experiments.
• EOS archive files are being repacked and separated from the backup tapepool, prior to migration to CTA.
• #271 Spacal.
• #274 FASER experiment on CTA

• AWAKE RQF1735636 - Archive 13 TB of AWAKE Run-1 data in /eos/experiment/awake Script to make FTS .json file.

• RQF1747988 CERN Digital Memory, #267

• RQF1790785 TOTEM files to copy from EOS. Everything ready from our side. We could help them by giving them the script to make the .json file.

14:40 → 14:45 AOB

• IT-ST-TAB round table