The CernVM Users Workshop will take place this year in the United Kingdom, hosted by Rutherford Appleton Laboratory in Oxfordshire with help from the STFC Scientific Computing Department and GridPP UK project.
Following the successful edition held at CERN in March 2015, the workshop aims to bring together users and developers to discuss the current status of the CernVM ecosystem and the future directions. Particular focus will be given to the integration with the containers world and how this can improve the experience with the cloud paradigm shift now occurring in HEP computing centers.
As a novelty, the third day of the workshop will be entirely dedicated to 'hands-on' sessions. Whilst the agenda is yet to be decided, there will be opportunities for demos and tutorials, also for users to come forward with topics for discussion.
A video-conference service will be provided to help remote attendance.
For questions or comments, please contact us at firstname.lastname@example.org.
This talk will present the status and plans of the CERN IT's CVMFS Operations Service, recently reassigned to our IT Storage Group. The first part of the talk will outline the architecture of the stratum zero, CERN stratum one, local squids; the notable feature of our service is that during 2015 is has been (almost) entirely moved to CERN's OpenStack infrastructure, making extensive use of Ceph and ZFS which has proven to be quite flexible while also offering a disaster recovery solution.
The second part of the talk will look to the future. CERN IT's recent change of strategy with respect to OpenAFS will put increased demands on CVMFS. We'll outline some of these new use-cases which are forthcoming for CVMFS, then present alternatives for the future scale-up and out of the system.
Existing research has shown the benefits of running multi-level schedulers, either for single node parallel computation or multi-node distributed computation. Apache Mesos is a 2-level distributed scheduling system that has been used in organizations such as Twitter, PayPal, and Apple.
In this talk we'll describe the 2-level Mesos architecture in detail. We'll also discuss Mesos features that have been added over the years (or are being added today) driven by practical needs, from weights, to reservations, to quotas, to optimistic allocations, and deallocation. In the second part we will focus on some of the operational characteristics of Mesos, as well as present the open-source Datacenter Operating System that leverages Mesos while providing capabilities such as service discovery, load balancing, privacy and ease of installation.
About the Speaker
Artem Harutyunyan is a Director of Engineering at Mesosphere where he works on Apache Mesos project. Prior to joining Mesosphere he spent several years at Qualys where he led the design and development of a distributed system for storing and analyzing large volumes of data. Prior to Qualys, Artem worked at CERN, first as student in ALICE Offline, and later as a fellow in CernVM group.
A short presentation on how and why HPC users are moving beyond public cloud for HPC workloads, including a discussion of achievable performance for both throughput and (briefly) MPI applications. We will also touch on compute accelerators, file system access, and containers, and provide some performance tuning tips for virtualizing HPC applications with high performance.
About the Speaker
With over 20 years of experience in High Performance and Technical Computing, Josh currently leads an effort at VMware to bring the value of virtualization to Research, Engineering, and Science environments. Previously, he was a Distinguished Engineer at Sun Microsystems with broad responsibilities for HPC direction and strategy. He joined Sun in 1996 from Thinking Machines Corporation, a pioneering company in the area of Massively Parallel Processors (MPPs), where he held a variety of technical positions. Josh has worked on developer tools for distributed parallel computing, including language and compiler design, scalable parallel debugger design and development, and MPI. He has also worked in the areas of 3D graphics, image processing, and real-time device control. Josh has an undergraduate degree in Engineering from Harvard College and a Masters in Computer Science from Harvard University. He has served as a member of the Board of Directors of OpenMP since 2002.
Cloud foundry is a PaaS that facilitates deployment and scale of web applications. It aims at providing a good user experience while ensuring that deployed application follow the best practices. Additionally, It provides a full set of features that enable debugging and monitoring of production systems. However, Docker is disrupting more traditional PaaS by making containers a prominent term in the cloud terminology. George, who works for Pivotal implementing Cloud Foundry's container runtime, will introduce Docker and Cloud Foundry, and discuss their differences. He will present the Docker image support in Cloud Foundry and eventually, how CernVM-FS could fit in the area of Docker image distribution.
A data federation is a cooperating set of storage resources transparently accessible across a wide area network via a common namespace. These are often implemented through a redirector hierarchy - clients query a centralized endpoint for a given file; this redirector locates an available storage resource, then redirects the client to the remote resource.
Data federations are an increasingly used as a way to distribute large-volumes of physics data. For example, the Compact Muon Solenoid (CMS) experiment has approximately 20PB of analysis data available through it's "Any Data, Any Time, Anywhere" (AAA) federation.
However, the namespace of AAA is extremely limited - it is equivalent to just a HTTP
GET. There are no directory listings, authoritative size or checksum information - despite the fact this information is known to CMS and available in the underlying storage systems and across several services; it is user-hostile for data discovery.
In this presentation, we will discuss a series of improvements made to the CVMFS core to marry a user-friendly, CVMFS-based POSIX namespace with data federation. We will demonstrate a set of CVMFS repositories of increasing complexity that utilize these new CVMFS features. These repositories serve as frontends for data federations for OSG, LIGO, and CMS.
Finally, we will discuss plans to grow this work - in terms of scale (data volume), efficiency, and features used in production.