CTA Dev Meeting

Europe/Zurich
513/R-068 (CERN)

513/R-068

CERN

19
Show room on map
Michael Davis (CERN)
Videoconference
CTA Dev Meeting
Zoom Meeting ID
68646510714
Description
CTA Dev Meeting
Host
Michael Davis
Useful links
Join via phone
Zoom URL
    • 3:00 PM 3:10 PM
      CTA Release Workflow 10m
    • 3:10 PM 3:20 PM
      CTA Release Roadmap 10m

      See CTA Release Roadmap

      Release 4.7.6-1

      • Milestones link
      • Release date: 24/06/2022
      • Pre-prod deployment date: 24/06/2022
      • Prod deployment date: 30/06/2022

      Release 4.7.7-1 [Catalogue v11]

      • Milestones link
      • Release date: 1 July
      • Pre-prod deployment date: 4 July
      • Prod deployment date: 6-8 July
      • Important: Avoid deployment on 4-5 July - High visibility event: Higgs 10th anniversary

      Release 4.7.8-1

      • Milestones link
      • Release date: TBD
      • Pre-prod deployment date: TBD
      • Prod deployment date: TBD
      • Will add support for multiple tape label formats

      Future releases (4.7.x)

      Public Release

      • We will swap releases 4.7.7-1 and 4.7.8-1, so that we can make the support for multiple tape label formats available earlier.
      • It will also allow us to fix immediately the tape drive regression issue introduced in 4.7.6 release.

      Planned schedule for 4.7.7-1:

      • Stress tests: Weekend, 2-3 Jul 2022
      • Release: Mon, 4 Jul 2022
      • PPS deployment: Mon, 4 Jul 2022
      • Prod deployment: Wed, 6 Jul 2022
    • 3:20 PM 3:30 PM
      Collaborations 10m

      CTA read support for non-native tape formats (OSM/dCache and Enstore)

      • Project roadmap: #1239 Add support to CTA for multiple tape label formats.
      • [Jorge - CTA team]: Refactor tape backend code, to allow new label formats to be implemented (#1257, #1263, ...). Will be released on v4.7.8, to ensure there are no regressions.

      cta-frontend-grpc integration for CTA

      • Project roadmap: #1236 Integrate cta-frontend-grpc into CTA repo and RPMs
      • [Michael - CTA team]: CTA Frontend protobuf changes to support dCache (#1240, #1244)
      • [Jacek - dCache]: Experimenting with streaming with gRPC (required for CTA admin client).

      cta-frontend-grpc integration for CTA:

      • Jacek will finish his tests with the async version of gRPC, and make it available in a separate branch, so that we can check it.
      • Sync version will only be created if deemed necessary after checking the async version,
    • 3:30 PM 3:40 PM
      HTTP REST API, Activities and Priorities 10m
      • Priority: Write document, with all specifications clearly defined, to be signed off by the experiments.
      • Topics requiring discussion:
        1) How will metadata be passed using HTTP REST API (the mechanism we use for XRootD will not work).
        2) What is the scope of what is meant by "Activities"? Is it for smart writing? Or for setting priorities? What is an "activity share" and what it is for?
        3) How will priorities be set in HTTP REST API? Using Activities? Or using the HTTP priorities mechanism?
      • Julien will take the lead in writing the document. It will be available by next week.
      • We decided to halt any work before the document is finished and approved by all the involved parties.
    • 3:40 PM 3:50 PM
      CTA dev topics 10m

      [IMPORTANT] Hole in recycle bin logic

      • Data loss incident due to operator error (#Ops-800).
      • Unable to recall all files due to hole in our recycle bin logic (#Ops-660).

      Tape drive states

      • [Vova]: Project status overview.
      • Related issue in release v4.7.6-1: #Ops-801

      PostgreSQL scheduler DB

      • [David]: Project status overview.
      • [Jorge]: Support David setting up the CI testing stages.

      REPACKING tape state and queue cleanup

      • [Joao]: Project status overview.

      Move subdirectories from CTA dev repo into new repos

      • [Richard]: Project status overview.
      • [All]: Feedback about the workflow proposed by Richard by the end of next week. Decide the next steps (check #1182)
      • [Fons/All]: SonarCloud usage for static code analysis - feedback.

      Retrieve queue youngest date error

      Other "needs discussion" topics

      Hole in recycle bin logic:

      • A potential solution for this issue had already been discussed in #1066 (logging the EOS metadata before deletion, so that it can be used when recovering a vile). Joao will increase the priority of this issue, so that a solution can be made available earlier.

      Tape drive states:

      • Vova found a solution for the bug introduced in release 4.7.6. This fix will be available in the next release 4.7.7.
      • We agreed not to revert the deployed CTA version to 4.7.5. In principle things should go ok during the weekend due to low activity and automatic recovery of eventual crashed processes.

      Move subdirectories from CTA dev repo into new repos:

      • Jorge will cleanup the auto-generated protobuf code that is poluting the git repo. We don't need this code to be pushed. It is also triggering the Sonarcloud checks unnecessarly.
    • 3:50 PM 4:00 PM
      AOB 10m

      Meeting Rooms

      • Room 513/R-068 is booked every week, until the EOY, for the CTA dev meeting.
      • There is a regression in EOS. Julien will talk will Elvin about a fix for the next EOS release.