Containers WG meeting - 17 October 2018

DRAFT for comments

Singularity status (Dave)

• See slides
• 3.0.0 alpha released - not ready yet.
• new implementation in Golang, if overlay off, falls back to underlay now
• new version doesn't have separate builder - could make patch for unprivileged RPM? Generally thought to be useful.
• several patches to address various things
• Won't put in EPEL until its more stable
• Sec review pending with Wisconsin, assume it takes 6 months, though less critical if a site is running unprivileged

OSG container CVMFS repo (Dave)

• See slides
• Docker to Singularity unpacked directory converter
• CMS and OSG is using this
• Scope is singualotl only - just unpacked directory
• GC an issue - drop images 30c days after no longer the current version
• Does EGI need one too? Having 3 seems not sensible.

ATLAS image usage (Alessandra)

• See slides
• Everything built up from layered docker images. Currently imported manually via script to CVMFS - would be helpful to have a service.
• ATLAS has both production images and user images.
• Potentially a very large number of user images, built up from standard layers. 5000 in gitlab already from early adopters.
• Both singularity and docker-layer approach deduplicate, can CVMFS handle that number of unpacked directory. Jakob thinks yes, though need to define GC policies.
• Really one (or multiple) image per user? Yes, users would like their own analysis container to ensure analysis reproducibility.
• Probably don't need any changes to CVMFS software to support user use-case, possibly looking at big release managers, and multiple to scale out the publication.

unpacked.cern.ch (Simone)

• See slides
• Prototype service in development, converting docker image to (CVMFS-hosted) flat singularity directory (as per OSG) and (CVMFS-hosted) docker layers to be served to clients using the CVMFS docker driver.
• Some discussion of potential directory structures and GC.
• Looks useful for ATLAS.
• Planning : keep in touch. So experiments can test it once we have a prototype service.