Kubernetes GitOps Workshop

Wednesday 27 Apr 2022, 09:00 → 17:00 Europe/Zurich

31/3-004 - IT Amphitheatre (CERN)

Ricardo Rocha (CERN)

Recording: https://videos.cern.ch/record/2295812

GitOps is a set of practices to manage infrastructure and application configurations using Git. It relies on Git as a single source of truth for declarative infrastructure and applications, and a set of reconcilers to apply changes.

Workshop goals:

• Assess the usage of GitOps in the Kubernetes/OpenShift community at CERN
• Share experiences on usage of the multiple GitOps tools, including ArgoCD, Flux (v1 and v2), GitLab CI or even custom deployment scripts around tools like Helm
• Consider consolidation of these efforts moving forward

If you are a user of such systems, please consider submitting an abstract with a title and a small description the tools and workflows you rely on to automate your deployments.

In addition to presentations there will be plenty of time for general discussion.

1 Welcome & Introduction

Speaker: Ricardo Rocha (CERN)

Kubernetes GitOps Workshop.pdf

2 RCS-SIS’ collaborations hosting

The CERN Scientific Information Service (RCS-SIS) participates in and supports multiple collaborations from the Open Science community. Some of these collaborative projects are hosted at CERN and run by the RCS-SIS Tools & Services team. To ensure a smooth development and release experience, the team rely entirely on GitOps principles and practices. All of this is made possible by using Github, Github Workflows & Actions, ArgoCD, Kustomize, Kubernetes and SealedSecrets.

Speaker: Benjamin Bergia (CERN)

RCS-SIS GitOps.pdf

3 Abusing helm library chart and values for fun and profit

After sharing some context details about how I manage my personal k3s cluster. I would share a different way to create helm charts that heavily relies on a library chart and makes everything configurable via helm values. This approach is the one that the k8s-at-home project uses (https://docs.k8s-at-home.com/our-helm-charts/common-library/).

Speaker: Arthur Outhenin-Chalandre (CERN)

codimd slides

4 GitOps in MONIT

MONIT follows the GitOps practices to run many internal services on Kubernetes.

Our first iteration involved Flux1 with Helm3, for which we accumulated some months of experience.

We recently migrated to Flux2 (still using Helm3), which we use in conjunction with a GitLab CI to manage several namespaces in multiple Kubernetes clusters for different environments (production, qa, development).

To provide some numbers: we currently manage 6 namespaces and 8 Kubernetes clusters (production, qa, and 6 namespace-specific ones for development). We run a total of 8 services and 18 cronjobs at the moment, and we also manage numerous PVs, PVCs, and secrets.

Our GitLab CI has two uses: the first one is a simple YAML validation; the second one is to automatically rebase some specific external commits (from a different automatic flow) from master to qa.

Secrets are managed through Mozilla’s SOPS and encryption keys generated with AGE.

We also have an extensive documentation which is slightly tailored towards MONIT-specific configurations and needs, which covers clusters setups, migrations, general Flux operations, and more.

Speaker: Luca Bello (CERN)

2022-04-27-GitOps-in-MONIT.pdf

2022-04-27-GitOps-in-MONIT.pptx

10:25 Coffee Break

5 K8s and GitOps for ATLAS Rucio operation

Recently, we migrated the Atlas Rucio installation from puppet to Kubernetes. This talk will presents how we use terraform, flux2 and helm to manage our multi-cluster installation and the applications inside the clusters.

Speaker: Radu Carpa (CERN)

atlas_rucio_k8s_gitops.pdf

6 GitOps for Web Frameworks: ArgoCD, Gitlab CI and custom operators

CERN's Web Frameworks are supported by a set of OpenShift clusters with shared components. We'd like to share how we use GitOps to manage their configuration, relying on ArgoCD, Helm charts and Gitlab CI. We'd also like to discuss how we can improve our current approach by relying more on ArgoCD, and upcoming solutions for secrets management.

The multiple supported versions of the CERN Drupal Distribution make new releases regularly, which need to be forwarded to the websites that use each version. We advertise new releases in Kubernetes and continuously deploy them with a Kubernetes Operator (controller + CRD) and Gitlab CI. We'd like to show you this particular custom use case, and discuss if it would be possible to use more standard components to achieve the same objective.

Speakers: Jack Henschel (CERN), Konstantinos Samaras-Tsakiris (CERN)

GitOps for Web Frameworks.pdf

7 The way of the force: GitOps on JEEDY

In this session you will learn how JEEDY team, build its internal infrastructure profiting of ArgoCD and git.
In particular we will show how we managed:

1. monitoring infrastructure for both us and users
2. submission of user cronjobs

The session will go through advantages, pitfalls of the GitOps approach that we faced during our development.

Speakers: Antonio Nappi (CERN), Ioannis Panagiotidis (Ministere des affaires etrangeres et europeennes (FR))

GitOps on JEEDY.pdf

JEEDY Slides

8 Shhh... It's a Secret!

There are many, many options to handle secrets in Kubernetes deployments, and even more when considering handling sensitive data in GitOps setups.

In this short presentation we present the differences between handling (encrypted) data in Git, secrets inside the clusters, and the different tools available to simplify these tasks.

Speaker: Ricardo Rocha (CERN)

Shhh… It’s a Secret!.pdf

9 Discussion: The Way Forward