10-14 October 2016
San Francisco Marriott Marquis
America/Los_Angeles timezone

An OSG-based distributed campus computing infrastructure

13 Oct 2016, 14:15
15m
Sierra B (San Francisco Mariott Marquis)

Sierra B

San Francisco Mariott Marquis

Oral Track 6: Infrastructures Track 6: Infrastructures

Speaker

Christoph Paus (Massachusetts Inst. of Technology (US))

Description

We describe the development and deployment of a distributed campus computing infrastructure consisting of a single job submission portal linked to multiple local campus resources, as well the wider computational fabric of the Open Science Grid (OSG). Campus resources consist of existing OSG-enabled clusters and clusters with no previous interface to the OSG. Users accessing the single submission portal then seamlessly submit jobs to either resource type using the standard OSG toolkit of HTCondor for job submission and scheduling and the CERN Virtual Machine File System (CVMFS) for software distribution. The usage of the Bosco job submission manager in HTCondor allows for submission to the campus HPC clusters without any access level beyond that of a regular user. The use of Condor flocking also allows user jobs to land at over a hundred clusters throughout the US that constitute the OSG Open Facility. We present the prototype of this facility, which enabled the Alpha Magnetic Spectrometer (AMS) experiment to utilize over 9 million computational hours in 6 weeks. We also present plans, including the usage of the new BoscoCE software stack to allow jobs submitted from outside the campus to land on any of the connected resources.

Primary Keyword (Mandatory) Computing facilities
Secondary Keyword (Optional) Distributed workload management

Primary authors

Dr Bo Jayatilaka (Fermi National Accelerator Lab. (US)) Christoph Paus (Massachusetts Inst. of Technology (US)) Dr Christopher Hill (MIT) Derek John Weitzel (University of Nebraska (US)) Maxim Goncharov (Massachusetts Inst. of Technology (US)) Yutaro Iiyama (Massachusetts Inst. of Technology (US))

Presentation Materials