14-18 October 2013
Amsterdam, Beurs van Berlage
Europe/Amsterdam timezone

Evaluating Google Compute Engine with PROOF

14 Oct 2013, 14:15
20m
Graanbeurszaal (Amsterdam, Beurs van Berlage)

Graanbeurszaal

Amsterdam, Beurs van Berlage

Oral presentation to parallel session Distributed Processing and Data Handling A: Infrastructure, Sites, and Virtualization Distributed Processing and Data Handling A: Infrastructure, Sites, and Virtualization

Speaker

Gerardo Ganis (CERN)

Description

The advent of private and commercial cloud platforms has opened the question of evaluating the cost-effectiveness of such solution for computing in High Energy Physics . Google Compute Engine (GCE) is a IaaS product launched by Google as an experimental platform during 2012 and now open to the public market. In this contribution we present the results of a set of CPU-intensive and I/O-intensive tests we have run with PROOF on a GCE resources made available by Google for test purposes. We have run tests on large scale PROOF clusters (up to 1000 workers) to study the overall scalability of coordinated multi-process jobs. We encountered the known scalability limitation of the PROOF technology in single master mode and we have investigated solution to lift this limitation. We have studied and compared the performance of ephemeral and persistent storage with PROOF-Lite on the single machines and of standard PROOF on the whole cluster. We will discuss our results in perspective, in particular with respect to the typical analysis needs of an LHC experiment.

Primary authors

Gerardo Ganis (CERN) Sergey Panitkin (Brookhaven National Laboratory (US))

Presentation Materials