Apr 12 – 16, 2010
Uppsala University
Europe/Stockholm timezone

A Business-Driven Cloudburst Scheduler for Bag-of-Task Applications

Apr 13, 2010, 4:20 PM
20m
Room IV (Uppsala University)

Room IV

Uppsala University

Oral Emerging technologies (cloud, virtualization etc) Novel Architectures and Technologies

Speaker

Prof. Francisco Brasileiro (UFCG)

Description

Opportunistic peer-to-peer (P2P) grid computing infrastructures have been proven to be cheap and effective platforms to run Bag-of-Tasks (BoT) applications. They operate on a best-effort model, which is understandable given their negligible cost. However, for some applications that could benefit from these infrastructures, having some guarantees on the time that takes to complete is an important requirement. In this work we investigate the possibility of combining a P2P grid with a cloud computing provider to provide some quality of service (QoS) guarantees.

Impact

The core of the scheduler is the module that decides when to buy new resources from the cloud provider. When an application is submitted for execution, the scheduler has no information about the QoS of the P2P grid, thus, it starts by trying to obtain as many machines as the number of tasks it has to run. At the end of each subsequent turn it computes the throughput of the P2P grid and estimates if the throughput needs to be enhanced by acquiring extra resources at the cloud provider. This scheduler was implemented as a broker of the OurGrid middleware and is instrumented to acquire extra resources from a cloud computer provider when needed. We currently have implementations for both Amazon EC2 and Eucalyptus. Results of measurement experiments show that the proposed scheduler is able to substantially increase the profitability of the infrastructure, in comparison with a traditional broker that does not consider the hybrid infrastructure. We also compared the cloudburst heuristic with an optimum scheduler and notice that the profitability of the former is close to the profitability achieved by the optimum scheduler.

Detailed analysis

We consider the problem of scheduling a BoT application in a hybrid computing infrastructure whose processing elements are comprised of in-house machines, virtual machines acquired from a cloud computing provider, and remote virtual machines made available by a best-effort P2P grid. The applications that run in this hybrid infrastructure are characterised by a utility function. The utility yielded by the completion of an application depends on the time taken to execute it. We take a business-driven approach to manage this infrastructure, aiming to maximise its profit, that is, the utility produced as a result of the applications that are run minus the cost of the computing resources that are used to run them. Each of these resources has a different cost basis. Whenever possible, applications are run using computing power just from the in-house capacity and from the P2P grid. Any extra capacity required to improve the profitability of the infrastructure is purchased from the cloud computing spot market on an on-demand basis. We propose a heuristic that is able to schedule BoT applications in this hybrid infrastructure.

Conclusions and Future Work

We are currently testing the cloudburst scheduler with the certification infrastructure of the EELA-2 project and an Eucalyptus-based cloud computing provider maintained by ourselves. Our next step is to identify potential users among the EELA-2 user community and the grid community at large.

URL for further information http://redmine.lsd.ufcg.edu.br/projects/show/ourgrid
Keywords cloud computing; opportunistic grid; peer-to-peer grid; bag-of-task applications

Author

Co-authors

Presentation materials