5th EGEE User Forum

Using the EELA-2 grid Infrastructure to Perform HeModynamics Computer Simulations: The HeMoLab Experience

12 Apr 2010, 17:57

3m

Aula (Uppsala University)

Poster Experiences from application porting and deployment Poster session

Speaker

Paulo G. P. Ziemer (LNCC - National Lab for Scientific Computing)

Description

The main goal of the HeMoLab project is the development of computational models used in the simulation of the Human Cardiovascular System, performed by a general purpose numerical solver that apply distributed computing techniques through the use of the MPI paradigm. This paper describes the main steps of the porting process of SolverGP to gLite environment, covers the main characteristics of gLite, grid tools that have been used in the gridification process, and examples of performance results of computer simulation that were obtained from the application already ported to Grid of the EELA-2.

Conclusions and Future Work

This work covers the main characteristics of gLite, the Grid tools that have been used in the gridification process and examples of performance results of computer simulation that were obtained from the application already ported to EELA-2. We will use the C++ API from the HeMoLab interface in order to create a more friendly way to submit jobs, run some larger cases (that take more than a week of computing time), make tests to discover the speedup of SolverGP, explore DIRAC capabilities (and others tools) and integrate our cluster in the EELA-2 infrastructure.

Detailed analysis

The simulation of the human cardiovascular system is performed using 1D and 3D models. The computer simulation is performed through the solution (using Finite Element Method) of a Navier Stokes equations system that describe the motion of fluids, such as blood inside of an artery. A set of numerical methods were implemented using Fortran 90 and the libraries PETSc and Parmetis in a framework called SolverGP, that uses MPICH2 as the MPI standard implementation.
We created a module between HeMoLab and EELA-2 infrastructure, in order to launch the remote script for execution of the SolverGP, encrypt the input files, trigger the capabilities for jobs monitoring and receive the output files generated remotely.
Before submitting a job, the encripted inputFiles, the SolverGP executable, and the HeMoLab libraries must be submitted. A JDL file is configured to define the files that will be sent directly to the Worker Nodes: the script of solver, the Watchdog (to enable the online monitoring) and the Secure Store (libraries required to decrypt the files).
After the calculation of the simulation, the files are moved from Grid Node to User Interface, in order to be sent for display in HeMoLab.

Impact

Some of our calculations need many days or a month for processing. Thus the HeMoLab project, along with the infrastructure of EELA-2, will provide simulating computational tools that can be used in the training of physicians and speeding up surgical planning. For instance, the computer model of an aorta, needs approximately 500,000 points to represent the artery geometry and then originates a system with 3,500,000 equations. The computing of just one heart beat period in such a model takes approximately 35 days (with 640 time steps) when using 8 processors in a 2X quadcore Xeon 3GHz 64GB RAM.
Despite the delay in the allocation of processors for the execution of jobs (in tests with 60 processors), we have noticed a decrease in processing time with respect to the tests used in our local cluster of 8 machines. Through the creation of the interoperability layer between HeMoLab and the services of the EELA-2, the processing power had a great increase. The increase from 36 to 715 processors will permit the speedup of SolverGP. Another advantage is the ability to run 8 jobs in a parallel way (number of EELA-2 sites that satisfy requirements to run SolverGP).