The DII-HEP OpenStack based CMS Data Analysis for secure cloud resources

Not scheduled
15m
OIST

OIST

1919-1 Tancha, Onna-son, Kunigami-gun Okinawa, Japan 904-0495
poster presentation Track7: Clouds and virtualization

Speaker

Mr Lirim Osmani (Department of Computer Science, University of Helsinki)

Description

The topic of data storage and analysis on Cloud infrastructures has gained importance in recent years. The High Energy Physics community is interested in performing simulations and data analysis on public or private Cloud facilities. Currently the simulations and analysis are performed mostly on a computing and data Grid. The software and experience of operating on a Grid needs to be adapted for running on Cloud facilities. The approach of harnessing Grid and Cloud technologies ensures a steady and seamless transition towards new ways of working. In order to fulfill this task, a virtual cluster has been constructed that is Cloud-based and Grid-enabled. For the Cloud part we use OpenStack and for the Grid software, that controls the execution of the physics jobs, we use components of the Advanced Resource Connector (ARC). This allows the end-users to submit the jobs with their preferred Grid or Cloud submission system and at the same time provides flexibility to maintain the infrastructure. The analysis software and libraries are installed via the CERN Virtual Machine file system. The cluster is monitored by running Site Availability Monitoring (SAM) jobs as well by using Graphite. Our solution uses a hybrid approach of combining elements of Cloud and Grid software components. To manage the virtual machines (VMs) dynamically in an elastic fashion, we are using the EMI authorization service (Argus) and the Execution Environment Service (Argus-EES) with an OpenStack plugin that has been developed for Argus-EES. Our newly developed plugin for the Argus-EES can communicate with multiple OpenStack deployments to expand and shrink resources dynamically upon demand. The Host Identity Protocol (HIP) has been designed for mobile networks and it provides a secure method for IP mobility and multi homing. HIP separates the end-point identifier and locator role for IP address which improves network agility of applications and the underlying virtual machines. Our solution leverages HIP for traffic management. This is useful for secure connections of hybrid Cloud resources. We describe the state and current experience with a virtualized computing environment for CMS data analysis in the Datacenter Indirection Infrastructure for Secure High Energy Physics (DII-HEP) project.

Primary author

Mr Lirim Osmani (Department of Computer Science, University of Helsinki)

Co-authors

John White White (Helsinki Institute of Physics (FI)) Dr Miika Komu (Ericsson Research) Prof. Paula Eerola (University of Helsinki (FI)) Dr Salman Toor (Helsinki Institute of Physics (FI)) Prof. Sasu Tarkoma (Department of Computer Science, University of Helsinki) Tomas Linden (Helsinki Institute of Physics (FI))

Presentation materials