Speakers
Description
CERN's computing infrastructure manages thousands of services across a complex distributed environment, requiring robust secret management for application credentials, root accounts, certificates, and service tokens. This paper explores CERN's transition from puppet-oriented, in-house secrets management solutions to HashiCorp Vault as a centralized, enterprise-level secret management platform.
For over twelve years, CERN relied on its own Python-based tools using RIAK and database stores for secret management, oriented towards in the Puppet-managed world. These approaches lacked functionality as CERN's infrastructure evolved toward K8S clusters, OKD orchestration, and token-based workflows.
The migration to HashiCorp Vault addressed these limitations through a phased approach. We implemented a high-availability Vault cluster with integrated RAFT storage, onboarding new projects first, then migrating existing ones. Custom authentication backends integrated with CERN's existing systems, while purpose-built tooling automated legacy migration and established standardized deployment workflows.
Key milestones included onboarding the HTVault solution, managed with Puppet in CERN's central infrastructure, and application secrets provisioning for OpenShift Projects. Based on GitLab project descriptions, we developed automated per-project secrets management synchronized with CERN OIDC and LDAP.
Migration from RIAK presented challenges: re-modeling existing structures, projecting ACLs from the legacy system, and adapting CLI tools to work transparently with Vault. We are exploring additional Vault benefits, including CERN certificate deployment and renewal, replacing another legacy tool.
This paper covers the technical architecture, migration challenges, lessons learned, and provides a roadmap for organizations considering similar transitions in high-energy physics computing environments.