19–25 Oct 2024
Europe/Zurich timezone

Cache Rules Everything Around Me: Building ePIC Containers With Spack

22 Oct 2024, 13:30
18m
Large Hall B

Large Hall B

Talk Track 7 - Computing Infrastructure Parallel (Track 7)

Speaker

Wouter Deconinck

Description

The ePIC collaboration is working towards the realization of the first detector at the upcoming Electron-Ion Collider. As part of our computing strategy, we have settled on containers for the distribution of our modular software stacks using spack as the package manager. Based on abstract definitions of multiple mutually consistent software environments, we build dedicated containers on each commit of every pull request for the software projects under our purview. This is only possible through judicious caching from container layers, over downloaded artifacts and binary builds, down to individual compiled files. These containers are subsequently used for our benchmark and validation workflows. Our container build infrastructure runs with redundancy between GitHub and self-hosted GitLab resources, and can take advantage of cloud-based resources in periods of peak demand. In this talk, I will discuss our experiences with newer features of spack, including storing build products as OCI layers and inheritance of previously concretized environments for software stack layering.

Primary author

Presentation materials