10–14 Oct 2016
San Francisco Marriott Marquis
America/Los_Angeles timezone

Interfacing HTCondor-CE with OpenStack

12 Oct 2016, 12:00
15m
Sierra C (San Francisco Mariott Marquis)

Sierra C

San Francisco Mariott Marquis

Oral Track 7: Middleware, Monitoring and Accounting Track 7: Middleware, Monitoring and Accounting

Speaker

Jose Caballero Bejar (Brookhaven National Laboratory (US))

Description

Over the past few years, Grid Computing technologies have reached a high
level of maturity. One key aspect of this success has been the development and adoption of newer Compute Elements to interface the external Grid users with local batch systems. These new Compute Elements allow for better handling of jobs requirements and a more precise management of diverse local resources.

However, despite this level of maturity, the Grid Computing world is
lacking diversity in local execution platforms. As Grid
Computing technologies have historically been driven by the needs of the High Energy Physics community, most resource providers run the platform (operating system version and architecture) that best suits the needs of their particular users.

In parallel, the development of virtualization and cloud technologies has accelerated recently, making available a variety of solutions, both
commercial and academic, proprietary and open source. Virtualization facilitates performing computational tasks on platforms not available at most computing sites.

This work attempts to join the technologies, allowing users to interact
with computing sites through one of the standard Computing Elements, HTCondor-CE, but running their jobs within VMs on a local cloud platform, OpenStack, when needed.

The system will re-route, in a transparent way, end user jobs into dynamically-launched VM worker nodes when they have requirements that cannot be satisfied by the static local batch system nodes. Also, once the automated mechanisms are in place, it becomes straightforward to allow an end user to invoke a custom Virtual Machine at the site. This will allow cloud resources to be used without requiring the user to establish a separate account. Both scenarios are described in this work.

Primary Keyword (Mandatory) Virtualization
Secondary Keyword (Optional) Computing middleware
Tertiary Keyword (Optional) Computing facilities

Author

Jose Caballero Bejar (Brookhaven National Laboratory (US))

Co-authors

Brian Paul Bockelman (University of Nebraska (US)) John Hover (Brookhaven National Laboratory (BNL)-Unknown-Unknown)

Presentation materials