CTA Dev Meeting

Friday 1 Jul 2022, 15:00 → 16:35 Europe/Zurich

513/R-068 (CERN)

Michael Davis (CERN)

15:00 → 15:10 CTA Release Workflow

• Release procedure instructions

15:10 → 15:20 CTA Release Roadmap

See CTA Release Roadmap

Release 4.7.6-1

• Milestones link
• Release date: 24/06/2022
• Pre-prod deployment date: 24/06/2022
• Prod deployment date: 30/06/2022

Release 4.7.7-1 [Catalogue v11]

• Milestones link
• Release date: 1 July
• Pre-prod deployment date: 4 July
• Prod deployment date: 6-8 July
• Important: Avoid deployment on 4-5 July - High visibility event: Higgs 10th anniversary

Release 4.7.8-1

• Milestones link
• Release date: TBD
• Pre-prod deployment date: TBD
• Prod deployment date: TBD
• Will add support for multiple tape label formats

Future releases (4.7.x)

• Milestones link

Public Release

• Remaining tasks

• We will swap releases 4.7.7-1 and 4.7.8-1, so that we can make the support for multiple tape label formats available earlier.
• It will also allow us to fix immediately the tape drive regression issue introduced in 4.7.6 release.

Planned schedule for 4.7.7-1:

• Stress tests: Weekend, 2-3 Jul 2022
• Release: Mon, 4 Jul 2022
• PPS deployment: Mon, 4 Jul 2022
• Prod deployment: Wed, 6 Jul 2022

15:20 → 15:30 Collaborations

CTA read support for non-native tape formats (OSM/dCache and Enstore)

• Project roadmap: #1239 Add support to CTA for multiple tape label formats.
• [Jorge - CTA team]: Refactor tape backend code, to allow new label formats to be implemented (#1257, #1263, ...). Will be released on v4.7.8, to ensure there are no regressions.

cta-frontend-grpc integration for CTA

• Project roadmap: #1236 Integrate cta-frontend-grpc into CTA repo and RPMs
• [Michael - CTA team]: CTA Frontend protobuf changes to support dCache (#1240, #1244)
• [Jacek - dCache]: Experimenting with streaming with gRPC (required for CTA admin client).

cta-frontend-grpc integration for CTA:

• Jacek will finish his tests with the async version of gRPC, and make it available in a separate branch, so that we can check it.
• Sync version will only be created if deemed necessary after checking the async version,

15:30 → 15:40 HTTP REST API, Activities and Priorities

• Priority: Write document, with all specifications clearly defined, to be signed off by the experiments.
• Topics requiring discussion:
  1) How will metadata be passed using HTTP REST API (the mechanism we use for XRootD will not work).
  2) What is the scope of what is meant by "Activities"? Is it for smart writing? Or for setting priorities? What is an "activity share" and what it is for?
  3) How will priorities be set in HTTP REST API? Using Activities? Or using the HTTP priorities mechanism?

• Julien will take the lead in writing the document. It will be available by next week.
• We decided to halt any work before the document is finished and approved by all the involved parties.

15:40 → 15:50 CTA dev topics

[IMPORTANT] Hole in recycle bin logic

• Data loss incident due to operator error (#Ops-800).
• Unable to recall all files due to hole in our recycle bin logic (#Ops-660).

Tape drive states

• [Vova]: Project status overview.
• Related issue in release v4.7.6-1: #Ops-801

PostgreSQL scheduler DB

• [David]: Project status overview.
• [Jorge]: Support David setting up the CI testing stages.

REPACKING tape state and queue cleanup

• [Joao]: Project status overview.

Move subdirectories from CTA dev repo into new repos

• [Richard]: Project status overview.
• [All]: Feedback about the workflow proposed by Richard by the end of next week. Decide the next steps (check #1182)
• [Fons/All]: SonarCloud usage for static code analysis - feedback.

Retrieve queue youngest date error

• [Vlado]: Issue #Ops-796

Other "needs discussion" topics

• Issues link

Hole in recycle bin logic:

• A potential solution for this issue had already been discussed in #1066 (logging the EOS metadata before deletion, so that it can be used when recovering a vile). Joao will increase the priority of this issue, so that a solution can be made available earlier.

Tape drive states:

• Vova found a solution for the bug introduced in release 4.7.6. This fix will be available in the next release 4.7.7.
• We agreed not to revert the deployed CTA version to 4.7.5. In principle things should go ok during the weekend due to low activity and automatic recovery of eventual crashed processes.

Move subdirectories from CTA dev repo into new repos:

• Jorge will cleanup the auto-generated protobuf code that is poluting the git repo. We don't need this code to be pushed. It is also triggering the Sonarcloud checks unnecessarly.

15:50 → 16:00 AOB

Meeting Rooms

• Room 513/R-068 is booked every week, until the EOY, for the CTA dev meeting.

• There is a regression in EOS. Julien will talk will Elvin about a fix for the next EOS release.