CTA Dev Meeting

Friday 6 May 2022, 14:00 → 15:30 Europe/Zurich

31/S-023 (CERN)

Michael Davis (CERN)

14:00 → 14:10 CTA Catalogue Schema

Next CTA Schema Version

• Version v10 backwards-incompatible changes #1151
• Most schema changes are now in master
• Need to add additional foreign key constraints to DISK_INSTANCE table (VO, ARCHIVE_FILE and the 3 mount rules tables)—will be done after Easter.
• Review of schema upgrade script/Liquibase
• Will we need a further schema change to remove deprecated columns?

Schema and software upgrades

• Need a process to better evaluate dependencies between software changes and schema changes and order of software/schema upgrades #1129
• Proposal to not bundle non-backwards-compatible schema changes with other code changes #706

• The clone of the production DB is ready to be used to test the CTA schema upgrade. Julien will take care of these tests.
• There are 2 clone instances available for future use:
  • Each with 230GBs, taking 2h30 to setup.
  • We can recreate and use them, as needed, for testing purposes.
• These test upgrades will tell us how long it takes to upgrade the production DB.
• Tests are planned to finish by Monday. Production DB upgrade scheduled for Thursday.

14:10 → 14:20 CTA Release Roadmap

See CTA Release Roadmap

Roadmap Status

Release 4.7.0

• Deployed in PPS, will be deployed in production next week (Thu 12 May)

Release 4.7.1

• Code clean-up release
• Clean up git repo #1182
• Update man pages #979, #1093

Release 4.7.x

• DB schema clean-up (remove obsolete columns and tables)
• Tape Server Drive/session status (Vova)
• Repack tape state

Deployment Schedule

• Release 4.6.3-1 / DB schema v4.6 has been deployed in production
• Release 4.7.0-1 / DB schema v10.0 has been deployed in PPS
• w/c 9/05/2022: Test DB schema upgrade on clone of production DB
• Thu 12/05/2022: Deploy schema v10.0 and CTA v4.7.0
• No date set yet for v4.7.1 code freeze

• Miguel wants to include cta-1198 changes (use hardcoded mount policy for verification requests) on release 4.7.1. Merge request is ready.
• Tag can be applied afterwards, marking the 4.7.1 release target.

 

With Jacek (dCache) we discussed:

• It's safe for him to upgrade their CTA instances to 4.7.0. This is already working in Pre-production at CERN.
• However, we recommended him to wait until our release of 4.7.0 is done in Production. This will guarantee that all our instructions/documentation have been upgraded, and that the containers used for the DB upgrade have been properly improved.

Other:

• Vova's tape server changes will included in a later release (4.7.2), to be isolated from other major changes.

14:20 → 14:30 Repack/Disabled Tape Statuses

Joao will present the current status, proposed logic and open questions which should be discussed.

Some notes:

• New runner on maintenance process will take care of "cleaning" requests on tapes moving into REPACKING/BROKEN state.
  • If a copy exists on a different tape the requests needs to be moved. Otherwise, it should be reported back to the user as an error.
• New internal states REPACKING_PENDING/BROKEN_PENDING indicate that the "cleaning" process is in progress. This state is set by the cta-admin command.
• Cta-admin command also sets up a flag in the queue object. This flag will be checked by the maintenance process in order to know when the "cleaning" process should start.
• Cta-admin must create an empty queue object if the object did not exist before. This is needed to avoid race condition. It only applies when setting REPACKING/BROKEN states.

Todo:

• Joao will upgrade the ticket with all the discussed details.

14:30 → 14:40 CASTOR to CTA Migration

CASTOR clean-up tasks #677

• Migrate CASTOR statistics to CTA
• Vlado to move supply tapes etc. to CTA
• Giuseppe to eventually remove CERNBox files from Hadoop tapes in the r_backup pool so they can be repacked. (No deadline has been set).

14:40 → 14:50 AOB

• dCache tape format integration
• Problems caused by ambiguous resolution of partial template specialisations. Latest example (though not the first) is #1194