See https://twiki.cern.ch/twiki/bin/view/LCG/WLCGContainers for working group page and actions. Agreed baseline doc is here.

WLCG Containers Working Group

Europe/Zurich
513/R-070 - Openlab Space (CERN)

513/R-070 - Openlab Space

CERN

15
Show room on map

Containers WG meeting - 17 October 2018

DRAFT for comments

 

Singularity status (Dave)

  • See slides
  • 3.0.0 alpha released - not ready yet.
  • new implementation in Golang, if overlay off, falls back to underlay now
  • new version doesn't have separate builder - could make patch for unprivileged RPM? Generally thought to be useful.
  • several patches to address various things
  • Won't put in EPEL until its more stable
  • Sec review pending with Wisconsin, assume it takes 6 months, though less critical if a site is running unprivileged

 

OSG container CVMFS repo (Dave)

  • See slides
  • Docker to Singularity unpacked directory converter
  • CMS and OSG is using this
  • Scope is singualotl only - just unpacked directory
  • GC an issue - drop images 30c days after no longer the current version
  • Does EGI need one too? Having 3 seems not sensible.

 

ATLAS image usage (Alessandra)

  • See slides
  • Everything built up from layered docker images. Currently imported manually via script to CVMFS - would be helpful to have a service.
  • ATLAS has both production images and user images.
  • Potentially a very large number of user images, built up from standard layers. 5000 in gitlab already from early adopters.
  • Both singularity and docker-layer approach deduplicate, can CVMFS handle that number of unpacked directory. Jakob thinks yes, though need to define GC policies.
  • Really one (or multiple) image per user? Yes, users would like their own analysis container to ensure analysis reproducibility.
  • Probably don't need any changes to CVMFS software to support user use-case, possibly looking at big release managers, and multiple to scale out the publication.

 

unpacked.cern.ch (Simone)

  • See slides
  • Prototype service in development, converting docker image to (CVMFS-hosted) flat singularity directory (as per OSG) and (CVMFS-hosted) docker layers to be served to clients using the CVMFS docker driver.
  • Some discussion of potential directory structures and GC.
  • Looks useful for ATLAS.
  • Planning : keep in touch. So experiments can test it once we have a prototype service.

 

There are minutes attached to this event. Show them.
    • 16:00 16:05
      Review of minutes from previous meeting 5m

      See https://indico.cern.ch/event/739456/note/

      Speaker: Gavin McCance (CERN)
    • 16:05 16:20
      Singularity update and OSG singularity service 15m
      Speaker: Dave Dykstra (Fermi National Accelerator Lab. (US))
    • 16:20 16:35
      ATLAS container service needs 15m
      Speaker: Alessandra Forti (University of Manchester (GB))
    • 16:35 16:50
      unpacked.cern.ch CVMFS prototype service 15m
      Speakers: Jakob Blomer (CERN), Simone Mosciatti (Politecnico di Milano (IT))
    • 16:50 17:10
      [Agenda WIP] Discussion and testing 20m
    • 17:10 17:20
      Current status and review of actions 10m