10–14 Oct 2016
San Francisco Marriott Marquis
America/Los_Angeles timezone

Consolidation of Docker use in HTC and its evolution to HPC at INFN-Pisa

12 Oct 2016, 12:15
15m
Sierra B (San Francisco Mariott Marquis)

Sierra B

San Francisco Mariott Marquis

Oral Track 6: Infrastructures Track 6: Infrastructures

Speaker

Enrico Mazzoni (INFN-Pisa)

Description

Clouds and Virtualization are typically used in computing centers to satisfy diverse needs: different operating systems, software releases or fast servers/services delivery. On the other hand solutions relying on Linux kernel capabilities such as Docker are well suited for applications isolation and software developing. In our previous work (Docker experience at INFN-Pisa Grid Data Center*) we discussed the possibility to move an HTC environment such as a CMS Tier2 into a Docker approach. During last year we have consolidated the use of Docker for the HTC part of our computing center. Our computing resources are leveraging a two levels infrastructure. The bare metal servers are operated by a Linux operating system compliant with the hardware (HCA, CPU, etc) and software (typically filesystems) requirements. This is the first layer and defines the system administrator's domain. The second layer takes care of users' needs and is administered with Docker. This approach improves the isolation of the user domain from the system administrator domain. It also increases the standardization of systems thus reducing the time needed to put into production.
Given the success with the HTC environment we decided to extend this approach to the HPC part of the INFN-Pisa computing center. Up to now about 25% of our HPC resources are using Docker for the users' domain. We also decided to simplify the bare metal servers management for this we have started to evaluate the integration of the Docker approach with cluster management tools such as Bright Cluster Manager, one of the HPC market leaders of such tools. In this work we will describe the evolution of our infrastructure from HTC to HPC and the integration of Docker and Bright. Given that the use of Docker in our computer center has become more and more important, it was necessary to develop certain ancillary services starting from the image management. For this purpose we have installed an image repository service based on Portus. This service has been integrated with the INFN AAI. Also this aspects are discussed in this work.

*J. Phys.: Conf. Ser. (JPCS), Volume 664, 2015: http://iopscience.iop.org/article/10.1088/1742-6596/664/2/022029

Primary Keyword (Mandatory) Virtualization
Secondary Keyword (Optional) High performance computing

Primary author

Enrico Mazzoni (INFN-Pisa)

Co-authors

Alberto Ciampa (Universita degli Studi di Pisa-INFN, Sezione di Pisa) Dr Antonino Formuso (INFN-Pisa) Silvia Arezzini (INFN Italy)

Presentation materials