See https://twiki.cern.ch/twiki/bin/view/LCG/WLCGContainers for working group page and actions. Agreed baseline doc is here.

WLCG Containers Working Group

Europe/Zurich
513/R-070 - Openlab Space (CERN)

513/R-070 - Openlab Space

CERN

15
Show room on map

Containers WG meeting - 17 October 2018

DRAFT for comments

 

Singularity status (Dave)

  • See slides
  • 3.0.0 alpha released - not ready yet.
  • new implementation in Golang, if overlay off, falls back to underlay now
  • new version doesn't have separate builder - could make patch for unprivileged RPM? Generally thought to be useful.
  • several patches to address various things
  • Won't put in EPEL until its more stable
  • Sec review pending with Wisconsin, assume it takes 6 months, though less critical if a site is running unprivileged

 

OSG container CVMFS repo (Dave)

  • See slides
  • Docker to Singularity unpacked directory converter
  • CMS and OSG is using this
  • Scope is singualotl only - just unpacked directory
  • GC an issue - drop images 30c days after no longer the current version
  • Does EGI need one too? Having 3 seems not sensible.

 

ATLAS image usage (Alessandra)

  • See slides
  • Everything built up from layered docker images. Currently imported manually via script to CVMFS - would be helpful to have a service.
  • ATLAS has both production images and user images.
  • Potentially a very large number of user images, built up from standard layers. 5000 in gitlab already from early adopters.
  • Both singularity and docker-layer approach deduplicate, can CVMFS handle that number of unpacked directory. Jakob thinks yes, though need to define GC policies.
  • Really one (or multiple) image per user? Yes, users would like their own analysis container to ensure analysis reproducibility.
  • Probably don't need any changes to CVMFS software to support user use-case, possibly looking at big release managers, and multiple to scale out the publication.

 

unpacked.cern.ch (Simone)

  • See slides
  • Prototype service in development, converting docker image to (CVMFS-hosted) flat singularity directory (as per OSG) and (CVMFS-hosted) docker layers to be served to clients using the CVMFS docker driver.
  • Some discussion of potential directory structures and GC.
  • Looks useful for ATLAS.
  • Planning : keep in touch. So experiments can test it once we have a prototype service.

 

There are minutes attached to this event. Show them.