CTA Dev Meeting

Friday 16 Sept 2022, 15:00 → 16:35 Europe/Zurich

513/R-068 (CERN)

Michael Davis (CERN)

15:00 → 15:10 CTA Release Workflow

• Release procedure instructions

15:10 → 15:20 CTA Release Roadmap

See CTA Release Roadmap

Release 4.7.10-1 - Catalogue v12.0

• Milestones link
• Release date: 12 Sep
• Pre-prod deployment date: 16 Sep
• Prod deployment date: 19 Sep
• New states (EXPORTED, etc.) and index changes.

Release 4.7.11-1

• Milestones link
• Release date: 16 Sep
• Pre-prod deployment date: 22 Sep
• Prod deployment date: 23 Sep
• "Flush" all the latest main commits into a code release.
• Stress tests running.

Release 4.8.0-1

• Milestones link
• Release date: TBD
• Pre-prod deployment date: TBD
• Prod deployment date: TBD
• v4.8.0-0.rc1 has been released and deployed in preproduction for testing purposes: #ops-842

Future releases (4.7.x)

• Milestones link

Public Release

• Remaining tasks
• Release date: Sep 2022 (delayed)
• On top of internal release v4.7.11
• Include Xrootd 5
• Versioning same as internal release, but with v5 (v5.Y.Z)

15:20 → 15:30 Collaborations

CTA read support for non-native tape formats (OSM/dCache and Enstore)

• Project roadmap: #2
• [Jorge]: Handling of the non-native formats by the CleanerSession: #122

cta-frontend-grpc integration for CTA

• Project roadmap: #38
• [Michael] Frontend refactoring.
• [Joao] btime and other parameters to be filled by EOS: #145

CTA read support for non-native tape formats (OSM/dCache and Enstore)

• Eric asked for an update from Jorge, about the current status of the CleanerSession fix

cta-frontend-grpc integration for CTA

• Jacek has completed a proff of concep implementation.
• The next steps are:
  1. Integrate cta-frontend-grpc with the common code (to be shared by all cta-frontend implementations) that was developed recently by Michael.
  2. Add support for the cta-admin commands that to not require response (no streaming).
  3. Pick one cta-admin streaming command (for example tape ls) and implement it as proof of concept.

15:30 → 15:40 CTA dev topics

Tape drive states

• [Vova]: Project status overview.

Problem with m_maxBatchBytes logic in RecallTaskInjector

• [Vova]: Issue #150

PostgreSQL scheduler DB

• [David]: Project status overview.

cta-admin schedulinginfo subcommand

• [Michael] Current implementation removed. See #126.
• Print the next tape(s) that will be mounted per logical tape library.
• ATM this is not useful because the VID is not included. With the PostgreSQL scheduler DB, we should be able to easily add this!

REPACKING tape state and queue cleanup

• [Joao]: Project status overview.
• New states (REPACKING, REPACKING_DISABLED, EXPORTED, etc...) added in release v4.7.10-1: Diagram link

EOS report logs

• [Joao]: Project status overview. Changes merged to EOS #8.

cta-admin build for CentOS Stream-8, and other CI changes

• [Jorge]: Project status overview #72

archiveretrieve-eos5 failure

• [David] Current status of #129 debugging. This seems to have been impacting our CIs for a while.

Proposal to change EOS disk fid to fxid

• [Michael] Present and discuss solution to #134.

Encryption key script optimization (cta-get-encryption-key)

• Issue description #Ops-839. Main tasks:
• [Joao]: Add the column Key ID to the table TAPE_POOL #137
• [Richard]: Modify cta-get-encryption-key script to accept the Key ID, instead of using cta-admin #138
• Modify Tape Server code to pass the Key ID to the script #139

"needs discussion" topics

• Issues link

Tape drive states

• Vova is currently working on less intrusive tests.

Problem with m_maxBatchBytes logic in RecallTaskInjector #150

• The priority is to fix the bug that causes tapes to be dismounted before all the batches have been written.
• We should also request the requests by number of available slots, not the size of files.

cta-admin schedulinginfo subcommand

• Has been done and will not be touched for a long while (at least until Postgres scheduler is done). Remove this from discussion topics.

EOS report logs

• It has been merged to EOS. Joao needs to discuss with Julien how this will be used in CTA.

archiveretrieve-eos5 failure

• Remove this from next agenda.

Proposal to change EOS disk fid to fxid

• Always use hexadecimal value, as discussed in ticket.

Encryption key script optimization (cta-get-encryption-key)

• Richard with work independently on the script changes. They key --keyid will be used as the option to pass the key instead of the vid.
• Vova will work on the tape server side changes.

15:40 → 15:50 AOB

HTTP REST API, Activities and Priorities

• [Julien]: Specification document status and remarks.

Other

• Room 513/R-068 is booked every week, until the EOY, for the CTA dev meeting.

Feature toggles (aka feature flags)

• Joao offered to write a one-page document explaining the benefits of this development technique, and proposing a way of using it in our section. It should contain the required effort to implement.