Description
The development and new discoveries of a new generation of high-energy physics cannot be separated from the mass data processing and analysis. The BESIII experiments studies physics in the tau-charm energy region from 2GeV to 4.6 GeV, at the Institute of High Energy Physics (IHEP) in Beijing, China, which is a typical data-intensive computing requiring mass storage and efficient computing resources. With the rapid growth of experimental data, the data processing system encounters many problems such as low resource utilization, migration complex and so on, which makes it urgent to enhance the data analysis system ability. Cloud computing which uses virtualization technology provides many advantages to solve these problems in a cost-effective way.
However, offline software design, resource allocation and job scheduling of BESIII experiment are all based on physical machine. To make use of the cloud computing resources such as Opensatck, the integration of Openstack and existing computing cluster is a key issue. In this contribution，we present an on-going work that aims to integrate openstack cloud resources into BESIII computing cluster for providing distributed compute clouds for the BES III physics experiment and supplying seamless combination. In particular, we discuss our design of the cloud scheduler which used to integrate OpenStack cloud resources into existing TORQUE and Maui cluster. In aspect of job scheduling, we adopt the thought of pull mode as described follows: when torque asking for launching new VMs in the cloud, the job agent residing in the vm will pull the suitable job to local machine and the job will be sent back after completed. It is transparent to users and they can continue to submit jobs using qsub command without knowing anything about cloud. Lastly, we report on our development work of adaptive job scheduling strategy to improve resource utilization and job processing efficiency.
| Primary Keyword (Mandatory) | Virtualization |
|---|---|
| Secondary Keyword (Optional) | Computing models |
