Ganga and Diane: powerful job management and resource
Presented by Dr. Patricia MENDEZ LORENZO, Dr. Jakub MOSCICKI, Dr. Andrew MAIER, Dr. Massimo LAMANNA, Mr. Anton LECHNER, Dr. Hurng-Chun LEE, Adrian MURARU on 23 Sep 2008 from 16:06 to 16:06
Session: Demos and Posters
Description of the activity
Ganga is a lightweight job management tool developed at CERN, key component in the distributed Data Analysis for HEP. It interfaces to a variety of Grid and non-Grid backends using the same UI. DIANE provides a parallelization framework for applications and an optimizer of computing resources. It was developed at CERN to help VO using the distributed computing infrastructures more efficiently. The combination of both tools helps many Grid communities to use the Grid in an efficient way
Ganga has gained widespread use in HEP applications and beyond. All these communities have different requirements and the main challenge is the creation of a standard software infrastructure for the immersion of these communities onto the Grid. This general infrastructure is effectively "shielding" the applications from the details of the Grid. It is flexible and general enough to match the requirements of the different productions without including major changes in the design of the tool. Ganga supports a large number of backends. DIANE was used in a number of computing activities, such as Avian Flu, Geant4 testing, medical simulations, computing support for ITU etc. In Avian Flu Data Challange the utilization of computing resources controlled by DIANE was 3-5 times more efficient than with direct job submission with less human supervision. In the ITU exercise around 200k short jobs (with convergence speed ranging between 1-1000sec) were executed in about 1 hour.
From January 2007 to mid-2008 Ganga has been used by around 1500 users and has been installed locally in more than 50 sites around the world. Recently also the educative aspect of Ganga has been recognized and Ganga has become a part of the official EGEE tools. Contrary to other portals or tools, Ganga is not limited to specific VOs or infrastructures allowing new users to quickly exploit the EGEE infrastructure. It also allows for the interoperability of various Grid backends. DIANE plays an important role in the gridification of applications: very often there is a large gap between traditional application environment and, rather peculiar, Grid environment. DIANE is customizable by design and therefore offers great flexibility in adapting various application scenarios into the Grid model.
We will show how an application is gridified and how a user can immediately profit from the Grid using Ganga and Diane . We demonstrate application cases outside of the initial scope of High energy Physics in which the tool has been developed. The demo will show the ease of this transition from the traditional submission, running on single machines or a local batch cluster to running on the full EGEE infrastructure, using a concrete example from Lattice QCD