CernVM Users Workshop

5 Mar 2015, 08:00 → 6 Mar 2015, 18:00 Europe/Zurich

30/7-018 - Kjell Johnsen Auditorium (CERN)

Gerardo Ganis (CERN), Jakob Blomer (CERN)

Started as R&D project in 2008, CernVM has evolved in an ecosystem providing all the building blocks for a virtualized analysis environment for High Energy Physics experiments. Some of its components have become critical services for the LHC experiments and the recent developments in terms of image building and contextualization aim to better adapt to and cope with the cloud paradigm shift now occurring in HEP computing centers.

This workshop aims to bring together users and developers to discuss the current status of the CernVM ecosystem and the future directions.

A video-conference service will be provided to help remote attendance.

Thursday 5 March

Status of the CernVM Ecosystem

Convener: CernVM team

1 Welcome and Introduction

Speaker: Gerardo Ganis (CERN)

Slides Ganis-Intro.pdf

2 Status and Roadmap of CernVM

Speaker: Jakob Blomer (CERN)

Slides cernvm.pdf

3 Status and Roadmap of CernVM-FS

Speaker: Rene Meusel (CERN)

Slides cvmfs_roadmap_rmeusel.pdf talk.key

4 CernVM-FS Interface to S3

Speaker: Dr Seppo Heikkila (CERN)

Slides Presentation

10:30 Coffee, Tea

Status of the CernVM Ecosystem

Convener: CernVM team

5 CernVM Web-API

Speaker: Ioannis Charalampidis (CERN)

Slides CernVM-WebAPI-2015-Workshop.pdf CernVM-WebAPI-2015-Workshop.pptx

6 Status and Roadmap of Vac and Vcycle

Speaker: Andrew McNab (University of Manchester (GB))

Slides mcnab-vac-vcycle-5mar15.pdf

7 CernVM elastic clusters and feedback from ALICE

Speaker: Dario Berzano (CERN)

Slides 20150305-dberzano-cernvm-elastic_clusters_alice_feedback.pdf

Video https://cds.cern.ch/record/2002402

12:30 Lunch

Feedback from experiments and users

Convener: CernVM team

8 Feedback from ATLAS

Speaker: Doug Benjamin (Duke University (US))

Slides ATLAS-cvmfs-cernvm-Mar2015.pptx ATLAS-cvmfs-cernvm-Mar2015.pptx.pdf

Video https://cds.cern.ch/record/2002401

9 Feedback from LHCb

Speaker: Ben Couturier (CERN)

Slides LHCb-CernVM-UW.odp LHCb-CernVM-UW.pdf

Video https://cds.cern.ch/record/2002400

10 Feedback from CMS

Speaker: Andrew David Lahiff (STFC - Rutherford Appleton Lab. (GB))

Slides CMS_CernVMWorkshop.pdf CMS_CernVMWorkshop.pptx

11 CernVM for CMS open data

Speaker: Kati Lassila-Perini (Helsinki Institute of Physics (FI))

Transparents CernVMUserWS05032015.pdf

Video https://cds.cern.ch/record/2002399

15:30 Coffee, Tea

Feedback from experiments and users

Convener: CernVM team

12 Feedback from OSG

Speaker: Dr Jose Caballero Bejar (Brookhaven National Laboratory (US))

Slides oasis_cvmfs_workshop.pdf

13 Feedback from Asia

Speaker: Felix.hung-te Lee (Academia Sinica (TW))

Slides cernVM_workshop-asgc_20150305v02.pdf

Video https://cds.cern.ch/record/2002398

14 Feedback from EGI

Speaker: Catalin Condurache (STFC - Rutherford Appleton Lab. (GB))

Slides Feedback_from_EGI.pdf Feedback_from_EGI.pptx

Video https://cds.cern.ch/record/2002397

15 CernVM: Notes from the GridPP new user engagement programme

Speaker: Tom Whyntie (University of London (GB))

Slides twhyntie_GridPP_DRN000032-v1-0_cern-vm-notes-1.pdf twhyntie_GridPP_DRN000032-v1-0_cern-vm-notes-1.pptx

16 Integrating VMs into WLCG

Speaker: Andrew McNab (University of Manchester (GB))

Slides mcnab-vm-wlcg-5mar15.pdf

17 WLCG Stratum 0

Speaker: Steve Traylen (CERN)

Slides CvmFS_Wlcg_Stratum0.pdf

Friday 6 March

Technological trends

Convener: CernVM team

18 Docker Containers and Their Emerging Roles in Distributed Applications

Docker has stormed into the IT landscape recently as a new abstraction for linux containers. Its ease of use is set to revolutionize the way we package applications, share and deploy them on compute clusters. In this presentation we will discuss Docker principles, illustrate the new application life cycle that Docker offers and give an overview of the booming Docker ecosystem. We will then introduce several container management tools aimed at supporting Docker based applications in clusters. Specifically we will present Kubernetes and show how Mesos is playing its part. With this new IT landscape emerging, we will present some ideas on how these new technologies might impact LHC experiments and their applications. About the author. **Sebastien Goasguen** built his first compute cluster in the late 90's when they were still called Beowulf clusters while working on his PhD; he has been working on making computing a utility since then. He has done research in grid computing and high performance computing and with the advent of virtualization moved to cloud computing in the mid 2000s. While at Purdue and Clemson he was involved in the OSG and TeraGrid project and worked on CERN LXCLOUD as a scientific associate in 2009 and 2010. He is currently a Senior Open Source Solutions Architect at Citrix, where he works primarily on the Apache CloudStack project, helping develop the CloudStack ecosystem.Sebastien is a project management committee member (PMC) of CloudStack and Apache libcloud and a member of the Apache Software Foundation, he focuses on the cloud ecosystem and has contributed to Knife-cloudstack, Eutester and Ansible. He is the author of O'Reilly 60 recipes for CloudStack and is currently writing the O'Reilly Docker cookbook.

Speaker: Sébastien Goasguen (Citrix)

Slides goasguen-cernvm-2015.pdf goasguen-cernvm-2015.pptx

Video https://cds.cern.ch/record/2002394

19 Personalized Opportunistic Computing for CMS at Large Scale

In recent years, the WLCG has evolved toward a model of model of centralized scheduling of largely dedicated clusters provided by each member institution, missing some opportunities to harness opportunistically available resources. This talk will present our recent experience in building Lobster, an opportunistic workload management system for the elements of the CMS collaboration at Notre Dame, where our prototype system scavenging cycles from a Tier-3 center runs on O(10K) cores, which is on the scale of the largest Tier-2 centers. I will discuss the challenges inherent in this environment, explain how Parrot, CVMFS, and Work Queue have evolved to meet these needs, and suggest ways to exploit emerging technologies such as containers within the context of scientific workflows. About the author. **Douglas Thain** is an Associate Professor of Computer Science and Engineering at the University of Notre Dame, where he designs large scale distributed computing systems to power the needs of advanced science and engineering research in fields such as bioinformatics, high energy physics, and molecular dynamics. His team publishes the Cooperating Computing Tools, a collection of open source software that provides application frameworks, workflow systems, and data access tools for clusters, clouds, and grids. Prof. Thain received the Ph.D. from the University of Wisconsin, where he contributed to the HTCondor high-throughput computing system.

Speaker: Prof. Douglas Thain (University of Notre Dame)

Slides lobster-cvmfs.pdf lobster-cvmfs.pptx

Video https://cds.cern.ch/record/2002393

10:10 Coffee, Tea

Technological trends

Convener: CernVM team

20 Hybrid cloud environments and networking on AWS

The presentation will be around AWS networking services and solutions to connect on premises data centers with the AWS Cloud, as well as infrastructure by code capabilities and strategies. About the author: **Giulio Soro** is a Solutions Architect within the EMEA Public Sector Team of Amazon Web Services. Previously he worked as Technical Project Manager in the Sardinian Public Sector, supervising regional datacenter management and private cloud computing projects. Giulio holds a PhD in Electronics Engineering and a Master of Science in Telecommunications Engineering from University of Cagliari.

Speaker: Giulio Soro (Amazon Web Services)

21 Big Data in the Cloud - Processing and Performance

This talk will cover how big data and big computation is being handled in the Cloud. It gives practical examples using GCP including tools that apply like Google BigQuery, DataFlow, Container Engine/Kubernetes, and ways to evaluate their performance using Google's open source benchmarking tool PerfKitBenchmarker. About the author. **Anthony F. Voellm** is currently leading the Google Cloud Performance Team and has a wide range of experience from kernel and database engines to graphics and automated image and map extraction from satellite images. Anthony is an avid inventor with 7 technology patents issued. In his current role at Google Anthony is focused on delivering Price Performance to existing products like Google Compute Engine and Google Cloud Storage while also innovating new offerings. Anthony holds a Master of Science from George Washington University, BA in Physics and a BS in Computer Science and Mathematics from the University of Vermont.

Speaker: Anthony Voellm (Google)

Video https://cds.cern.ch/record/2007711

12:30 Lunch

Technological trends

Convener: CernVM team

22 Distributed Systems: The Hard Problems

The focus of this presentation will be around some interesting issues that arise in distributed systems, and how Riak and applications that use it go about solving them. About the author. **Nicholas Bellerophon** works as a client services engineer at Basho Technologies, helping customers setup and run distributed systems at scale in the wild. He has also worked in massively multiplayer games, and recently completed a live scalable simulation engine. He is an avid TED-watcher with interests in many areas of the arts, science, and engineering, including of course high-energy physics.

Speaker: Nicholas Bellerophon (Basho Technologies)

Slides CERN_Deck.key CERN_Deck.pdf

Video https://cds.cern.ch/record/2002392

CernVM related activities

23 Volunteer Computing Projects at CERN

Speaker: Nils Hoimyr (CERN)

Slides volunteer-computing-cernvm.pdf

Video https://cds.cern.ch/record/2002391

24 The Data Bridge: Spanning trusted and untrusted environments

Speaker: Laurence Field (CERN)

Slides 2015DataBridgeCernVM.pdf 2015DataBridgeCernVM.pptx

Video https://cds.cern.ch/record/2002390

25 Long-Term Data Preservation

Speaker: Frank Berghaus (University of Victoria (CA))

Slides cernvm-workshop-march2015.pdf cernvm-workshop-march2015.pptx

26 CVMFS as a High Speed Filesystem for Auxiliary Data

Speaker: Dave Dykstra (Fermi National Accelerator Lab. (US))

Slides CernVMWorkshop150306.pdf

Video https://cds.cern.ch/record/2002386

16:15 Coffee, Tea

Discussion

To be defined

Convener: CernVM team

27 Wrap-up

Speaker: Jakob Blomer (CERN)

Slides wrapup.pdf

Video https://cds.cern.ch/record/2002406