A Business-Driven Cloudburst Scheduler for Bag-of-Task Applications
Presented by Prof. Francisco BRASILEIRO on 13 Apr 2010 from 16:20 to 16:40
Session: Novel Architectures and Technologies
Track: Emerging technologies (cloud, virtualization etc)
Opportunistic peer-to-peer (P2P) grid computing infrastructures have been proven to be cheap and effective platforms to run Bag-of-Tasks (BoT) applications. They operate on a best-effort model, which is understandable given their negligible cost. However, for some applications that could benefit from these infrastructures, having some guarantees on the time that takes to complete is an important requirement. In this work we investigate the possibility of combining a P2P grid with a cloud computing provider to provide some quality of service (QoS) guarantees.
We consider the problem of scheduling a BoT application in a hybrid computing infrastructure whose processing elements are comprised of in-house machines, virtual machines acquired from a cloud computing provider, and remote virtual machines made available by a best-effort P2P grid. The applications that run in this hybrid infrastructure are characterised by a utility function. The utility yielded by the completion of an application depends on the time taken to execute it. We take a business-driven approach to manage this infrastructure, aiming to maximise its profit, that is, the utility produced as a result of the applications that are run minus the cost of the computing resources that are used to run them. Each of these resources has a different cost basis. Whenever possible, applications are run using computing power just from the in-house capacity and from the P2P grid. Any extra capacity required to improve the profitability of the infrastructure is purchased from the cloud computing spot market on an on-demand basis. We propose a heuristic that is able to schedule BoT applications in this hybrid infrastructure.
The core of the scheduler is the module that decides when to buy new resources from the cloud provider. When an application is submitted for execution, the scheduler has no information about the QoS of the P2P grid, thus, it starts by trying to obtain as many machines as the number of tasks it has to run. At the end of each subsequent turn it computes the throughput of the P2P grid and estimates if the throughput needs to be enhanced by acquiring extra resources at the cloud provider. This scheduler was implemented as a broker of the OurGrid middleware and is instrumented to acquire extra resources from a cloud computer provider when needed. We currently have implementations for both Amazon EC2 and Eucalyptus. Results of measurement experiments show that the proposed scheduler is able to substantially increase the profitability of the infrastructure, in comparison with a traditional broker that does not consider the hybrid infrastructure. We also compared the cloudburst heuristic with an optimum scheduler and notice that the profitability of the former is close to the profitability achieved by the optimum scheduler.
We are currently testing the cloudburst scheduler with the certification infrastructure of the EELA-2 project and an Eucalyptus-based cloud computing provider maintained by ourselves. Our next step is to identify potential users among the EELA-2 user community and the grid community at large.
cloud computing; opportunistic grid; peer-to-peer grid; bag-of-task applications