21-25 May 2012
New York City, NY, USA
US/Eastern timezone

Managing Virtual Machine Lifecycle in CernVM Project

22 May 2012, 13:30
4h 45m
Rosenthal Pavilion (10th floor) (Kimmel Center)

Rosenthal Pavilion (10th floor)

Kimmel Center

Poster Distributed Processing and Analysis on Grids and Clouds (track 3) Poster Session

Speaker

Ioannis Charalampidis (Aristotle Univ. of Thessaloniki (GR))

Description

The creation and maintenance of a Virtual Machine (VM) is a complex process. To build the VM image, thousands of software packages have to be collected, disk images suitable for different hypervisors have to be built, integrity tests must be performed, and eventually the resulting images have to become available for download. In the meanwhile, software updates for the older versions must be published, obsolete images must be revoked, and the clouds that use them must be updated. Initially, in the CernVM project we used several commercial solutions to drive this process. In addition to the cost, the drawback of such an approach was lack of a common and coherent framework that would allow for full control of every step in the process and easy adaptation to new technologies (hypervisors, clouds, APIs). In an attempt to provide a complete lifecycle management solution for virtual machines, we collected a set of open-source tools, adapted them to our needs and combined them with our existing development tools in order to create an extensible framework that could serve as end to end solution for VM lifecycle management. This new framework is based on the Archipel Open Source Project and shares some of its main principles, namely, every component of the system is a stand-alone agent; the front-end is a stand-alone application; all of them communicate over the same messaging network based on the Extensible Message and Presence Protocol (XMPP). Each component of the framework can thus interact with each other in order to perform automated tasks and all of them can be managed from a single User Interface. The agents that manage the Hypervisor infrastructure, as well as the agents that deploy and monitor the Virtual Machines and the web-based user interface, are provided by the Archipel Project. In CernVM, we developed iBuilder, a tool to instrument VM images for almost all popular hypervisors. We integrated Tapper, an open-source tool that tests the resulted images, and we developed all the appropriate agents to control the software repositories and the previously mentioned tools. All these agents now allow us to continuously build and test development images. To complete the system, we plan to develop agents that will be capable of deploying contextualized CernVM images in various scenarios such as clouds that support the EC2 API Interface, or private/academic clouds using the native tools. Finally, a new lightweight front-end is under development aiming to provide access and complete control of the framework from the portable devices (smartphones and tablets). In this contribution we will present the details of this system, it's current status and future plans.

Summary

The maintenance of a Virtual Machine is a complex process that involves many software packages, many different phases and no standardized methodology.

For the CernVM project our first attempts to use several commercial solutions to drive this process failed as they were unable to provide a common and coherent framework that would allow for full control of every step. We therefore decided to combine existing open-source tools into an extensible framework that could serve as end-to-end solution for VM lifecycle management.

This new framework is based on the Archipel Project and shares some of its main principles. The whole system is stateless, there is no central server, and all the involved components plus the user interfaces are just inter-connected over the same messaging network. In this way, new components can be added on-the-fly, every component can communicate with each other to perform automated tasks, and all of them can be controlled from the same interface.

In this contribution we will present the architecture of this framework, our new tools that allow us to continuously build and test development images, some in-development highlights and our future plans.

Student? Enter 'yes'. See http://goo.gl/MVv53 yes

Primary author

Ioannis Charalampidis (Aristotle Univ. of Thessaloniki (GR))

Co-authors

Artem Harutyunyan (CERN) Dag Larsen (University of Bergen (NO)) Jakob Blomer (Ludwig-Maximilians-Univ. Muenchen (DE)) Predrag Buncic (CERN)

Presentation Materials