Conference on Computing in High Energy and Nuclear Physics

Scalable and multi-tenant Kubernetes ingress infrastructure

MON 31

21 Oct 2024, 15:18

57m

Exhibition Hall

Poster Track 7 - Computing Infrastructure Poster session

Speaker

Jack Henschel (CERN)

Description

Since 2016, CERN has been using the OpenShift Kubernetes Distribution to host a platform-as-a-service (PaaS). This service is optimized for hosting web applications and has grown to tens of thousands of individual websites. By now, we have established a reliable framework that deals with varied use cases: thousands of websites per ingress controller (8K+ hostnames), handling with long-lived connections (30K+ concurrent sessions) and high traffic applications (25TB+ per day).

This session will discuss:

• CERN's web hosting infrastructure based on OpenShift Kubernetes clusters;
• usage of open source and in-house developed software for providing a seamless user experience:
• integrations for registering hostnames (local DNS, LanDB, external providers)
• provisioning of certificates (automatic with external-dns / ACME HTTP-01, manual provisioning)
• access control policies and "connecting" different components with OpenPolicyAgent
  • enforcing unique hostnames across multiple Kuberenetes clustes
• strategies for setting up Kubernetes Ingress Controllers for multi-tenant clusters;
• methods for scaling and sharding ingress controllers according to application requirements (specifically HAProxy ingress controllers);