21-25 August 2017
University of Washington, Seattle
US/Pacific timezone

Supercomputers, Clouds and Grids powered by BigPanDA for Brain studies

Aug 22, 2017, 5:05 PM
20m
Auditorium (Alder Hall)

Auditorium

Alder Hall

Oral Track 1: Computing Technology for Physics Research Track 1: Computing Technology for Physics Research

Speaker

Alexandre Beche (Ecole polytechnique fédérale de Lausanne (CH))

Description

The PanDA WMS - Production and Distributed Analysis Workload Management System - has been developed and used by the ATLAS experiment at the LHC (Large Hadron Collider) for all data processing and analysis challenges. BigPanDA is an extension of the PanDA WMS to run ATLAS and non-ATLAS applications on Leadership Class Facilities and supercomputers, as well as traditional grid and cloud resources. The success of the BigPanDA project has drawn attention from other compute intensive sciences such as biology. In 2017, a pilot project was started between BigPanDA and the Blue Brain Project (BBP) of the Ecole Polytechnique Federal de Lausanne (EPFL) located in Lausanne, Switzerland. This proof of concept project is aimed at demonstrating the efficient application of the BigPanDA system to support the complex scientific workflow of the BBP which relies on using a mix of desktop, cluster and supercomputers to reconstruct and simulate accurate models of brain tissue.

In the first phase, the goal of this joint project is to support the execution of BBP software on a variety of distributed computing systems powered by PanDA. The targeted systems for demonstration include: Intel x86-NVIDIA GPU based BBP clusters located in Geneva (47 TFlops) and Lugano (81 TFlops), BBP IBM BlueGene/Q supercomputer (0.78 PFLops and 65 TB of DRAM memory) located in Lugano, the Titan Supercomputer with peak theoretical performance 27 PFlops operated by the Oak Ridge Leadership Computing Facility (OLCF), and Cloud based resources such as Amazon Cloud.

To hide execution complexity and simplify manual tasks by end-users, we developed a web interface to submit, control and monitor user tasks and seamlessly integrated it with the BigPanDA WMS system. The project demonstrated that the software tools and methods for processing large volumes of experimental data, which have been developed initially for experiments at the LHC accelerator, can be successfully applied to other scientific fields.

Primary authors

Alexandre Beche (Ecole polytechnique fédérale de Lausanne (CH)) Kaushik De (University of Texas at Arlington (US)) Dr Fabien Delalondre (Ecole polytechnique fédérale de Lausanne (CH)) Alexei Klimentov (Brookhaven National Laboratory (US)) Prof. Felix Schürmann (Ecole polytechnique fédérale de Lausanne (CH)) Dr Ruslan Mashinistov (Russian Academy of Sciences (RU))

Presentation materials

Peer reviewing

Paper