21st International Conference on Computing in High Energy and Nuclear Physics (CHEP2015)

Design and development of a Virtual Computing Platform Integrated with Batch Job System

Not scheduled

15m

OIST

poster presentation Track7: Clouds and virtualization

Speaker

Jingyan Shi (IHEP)

Description

Batch system is a common way for a local cluster to schedule jobs running on work nodes. In some cases, some jobs have to stay in queue without suitable work nodes while some job slots have to keep free without suitable jobs running. The reasons for such case might be various. One of the main reasons is that operating system running on the free work nodes is different from the one that jobs in queues wanted. Another reason to block jobs running on free job slots is the complicated job scheduling policies. Virtual machine has been proved to run some type of jobs with low penalty, and it could provide more flexible Operating system the job wanted. We designed and developed a virtual computing platform based on openstack and integrated with batch job system. The same submit command and job script could be used without any changes. The jobs submitted to the virtual computing platform would be run on virtual machines with the correct operating system. All of these are transparent to users. The platform is composed of four parts, Batch Info Collector, Job Matcher, Virtual Machine Manager and Virtual Job Manager. After consults with the info from Batch Info Collector and Virtual Machine Manager, Job Matcher would send request to the Virtual Machine Manager to start virtual machines for the jobs submitted to the virtual queue. All the jobs running on virtual machine are monitored and controlled by virtual job manager.