Speaker
Dr
Gen Kawamura
(International Cloud Cooperation)
Description
Grid computing enables deployments of large scale distributed computational infrastructures among different research facilities. It has been recently proposed that the Grid infrastructure be based on cloud computing. Provisioning systems and automated management frameworks using Cobbler, Rocks, Cfengine and Puppet are being successfully applied to many systems. Having implemented these new concepts in Grid sites within the Worldwide LHC Computing Grid (WLCG), they are solving many problems with huge world-wide collaborations and experiments, such as ATLAS and CMS.
Software provisioning like Cobbler and Rocks involves the process of selecting a target machine, a server, loading the different systems (bare operating systems, middlewares, and applications), and customizing individual components and configuring the cluster to make them ready for production. The provisioning system needs a variety of actions: for example, creating or changing boot images; specifying particular hardware and network parameters; and lastly, starting the machine, its softwares, Grid middlewares and high energy physics applications. Typically, because of the complexity of the systems, system administrators will perform these tasks using various provisioning and management tools. However, there is a lack of generic control tools for their huge collaborations that can adequately integrate and automate these tasks for the large scale Grid sites and local experimental systems in the WLCG environments. Therefore, even though fundamental components between the large Grid production site and the small experimental system are similar, every administrator needs to customize and optimize these different tools and administrative codes when the site needs to be built and merged with WLCG collaborations.
In terms of resource provisioning, system management, meta-scheduling and portability of the Grid clusters, the combination of Grid and cloud resources for deploying the WLCG computing infrastructure introduces new challenges and opportunities. We demonstrate a way of effective administration and collaboration by using typical Grid middlewares and various services: globus, TorquePBS, CREAMCE, dCache, StoRM and gLite middlewares. Multiple virtual Grid clusters have been prepared and the deployments show the effectiveness of a generalized approach gaining the portability of the grid clusters.
In the present paper, we argue for integrating the concept of a localized LHC Computing Grid in a testing and deployment framework with that of the so-called Portable-Grid system. Using virtualization and revision control technologies to share generic system management codes, procedures installing grid systems, required libraries, experimental codes and bare operating systems, the remotely prepared Grid systems can be immediately deployed into virtual images and executed under suitable hypervisors installed on target host platforms and grid clusters. This approach is allowing the instant construction of localized Grid sites in a scenario based on OpenStack cloud computing. Further, it enables a new paradigm applicable also to general experiments in physics and computer science.
Primary author
Dr
Gen Kawamura
(International Cloud Cooperation)
