Demo description:
Demo will show following features and functionality:
- graphical user environment for job submission, monitoring and other grid
operations
- running applications from different disciplines and communities
- running within MD platform batch and MPI applications
- running sequential and interactive applications
Two applications had been selected to present MD framework and mentioned above
features: parallel ANN training application, MAGIC Monte Carlo Simulation
Parallel ANN training application - Interactive application from CrossGrid
–(description of usecase in technical background section)
This application is used to train an Artificial Neural Network (ANN) using
simulated data for the DELPHI experiment. The ANN is trained to distinguish between
signal (Higgs bosson) events and background event (in the demo the background used
includes WW and QCD events). The evolution of the training can be monitored using
the
MD with a graphics presenting curent error, and 4 small graphics that show the ANN
value vs. an event variable (that can be selected by the user). The application is
compiled with MPICH-P4 for intracluster use and with MPICH-G2 for intercluster use.
This application uses the interactive input channel to let the user make a clean
stop
of the training (instead of killing the job), and also the possibility of resetting
the ANN weights to random values, to avoid local minima.
MAGIC Monte Carlo Simulation
The MAGIC Monte Carlos Simulation (MMCS) is one of the generic applications within
EGEE. As the simulation of extensive air showers initiated by high
energetic cosmic rays is very compute intensive, the MAGIC collaboration – together
with Grid resource centers from the EGEE project - migrate the MMCS application
within the last years to the EGEE infrastructure to speed up the production of the
simulations. To get enough statistics for a physics analysis, many jobs with the
same
input parameters but different random numbers needs to be submitted. The submission
tools from the MAGIC Grid are integrated in the Migrating Desktop and its underlying
infrastructure. Therefore all services und features of the Migrating Desktop like
Job
Monitoring, Data management, etc. can be used by members of the MAGIC virtual
organization.
Platform and services
Testbed:
- EGEE production, GILDA and CrossGrid testbed
Applications:
- MAGIC application running on EGEE,
- ANN interactive application running on CrossGrid testbed
Services:
- usage of following EGEE services:
- WMS: RB, LB, CE
- Data Management: SE, LCG-UTILS (Replica Manager)
- Information Index
- usage of following CrossGrid testbed services
- WMS: RB, LB, CE
- Data Management: SE, LCG-UTILS (Replica Manager)
- Information Index
Technical background:
A number of Grid middleware projects are working on user interfaces for interaction
with grid applications, however due to the dynamic and complex nature of the Grid,
it’s not easy to attract new users like ordinary scientists. To solve this problem
we
introduce the concept of Migrating Desktop which is a graphical, user oriented tool
that simplifies the use of the grid technology in the application area.
The Migrating Desktop (MD)is an advanced graphical user interface and a set of tools
combined with user-friendly outlook, similar to window based operating systems. It
hides the complexity of the grid middleware and allows to access grid resources in
an
easy and transparent way with special focus on interactive and parallel grid
applications. These applications are both compute- and data-intensive and are
characterised by the interaction with a person in a processing loop. MD can attract
new users by its features: easy to use, platform independed, available everywhere,
enables possibility to easily add new application that can be batch or interactive,
sequential or parallel. Thanks to its open architecture it can easily integrate
existing or incoming tools that for example supports grid operations or enables
collaborative work.
This research refers to three different grid projects: EU BalticGrid project, EU
CrossGrid project, and Progress (co-founded by Sun Microsystems and the Polish State
Committee for Scientific Research). As a key product of CrossGrid project, Migrating
Desktop has proved its usefulness in everyday work of users community.
Technical background
Platform overview
The aim of the Migrating Desktop is to provide scientists with a framework which
hides the details of most Grid services and allows working with grid application in
an easy and transparent way. The graphical user interface integrates and makes use
of
number of middleware and integrates the individual tools into a single product
providing a complete grid front-end. It is built on base of a mechanism for
discovering, integrating, and running modules called bundles based on the OSGi
specification. When the MD is launched, the users can work with environment composed
of the set of bundles. Usually a small tool is written as a single bundle, whereas a
complex tool has its functionality split across several bundles. A bundle is the
smallest unit of our platform that can be developed and delivered separately. Such
approach allows increasing functionality in an easy way without the need of
architecture changes.
The Migrating Desktop framework allows the user to access transparently the Grid
resources, run sequential or interactive, batch or MPI applications, monitoring and
visualization, and manage data files. MD provides a front-end framework for
embedding
some of the application mechanisms and interfaces, and allows the user to have
virtual access to Grid resources from other computational nodes.
The MD is a front end to The Roaming Access Server (RAS), which intermediates to
communication with different grid middleware and applications. The Roaming Access
Server offers a well-defined set of web-services that can be used as an interface
for
accessing HPC systems and services (based on various technologies) in a common and
standardised way. All communication bases on web services technology.
Our platform can work with different grid testbeds: based on LCG 2.3/2.4, LCG 2.6,
Progress 1.0. Due to its open system nature it can be easily ported to support other
testbeds.
Applications use cases
MAGIC Monte Carlo Simulation
The MAGIC Monte Carlos Simulation (MMCS) is one of the generic applications within
EGEE. As the simulation of extensive air showers initiated by high energetic
cosmic rays is very compute intensive, the MAGIC collaboration– together with
Grid resource centers from the EGEE project - migrate the MMCS application within
the
last years to the EGEE infrastructure to speed up the production of the simulations.
The simulation of the air showers requires the most computing time, e.g. a request
for a Monte Carlo sample of 1.0 million gamma-events would need around 1500
computing
hours on a standard CPU (2~MHz PentiumIV). This can be speeded up by using many
resources by parallelizing the application, if possible. Therefore the simulation of
a Monte Carlo sample is split in subjobs of 1000 events to run in parallel on
distributed Grid resources.The resulting 1000 data files are transferred and stored
on a dedicated Grid storage center automatically when a subjob is finished. When all
files are available, a program merges them to one single file that is processed by
the next program of the Monte Carlo workflow.
To track and manage the big number of jobs, a meta database containing information
about single jobs, their status and available data was set up. The metadata are
stored in a separate relational database combining information from the Grid domain
with data needed by MAGIC scientists. A Grid user requests a given number of Monte
Carlo events by writing this into the meta database, while a daemon process
regularly
submits smaller bunches of subjobs to the Grid resources. The current implementation
of the MMCS system does not require any additional software installation on Grid
resources.
The submission tools from the MAGIC Grid are integrated in the Migrating Desktop and
its underlying infrastructure. Therefore all services und features of the Migrating
Desktop like Job Monitoring, Data management, etc. can be used by members of the
MAGIC virtual organization.
Interactive Application (CrossGrid) – Parallel ANN training application.
The user launches the ANN job wizard from the MD Job Wizard menu or from an already
existing job shortcut. After filling all the necessary parameters in Job Wizard the
user submits the job. Once it is running the ANN plugin can be launched. In the
plugin the user can see a panel with four graphics representing the value of the ANN
for a subset of the training events (signal events in green and background events in
red) vs. several variables of the events. The user can change the selected variables
using the combo list at the bottom of the plugin window. At the right side the user
can see the graphic representing the evolution of the ANN training error vs the
training epoch. The plugin also includes three options: "reset weights" that resets
the values of the ANN weights to random, "Stop application" - the program goes out
of the training loop stopping the training and "Exit" for closing the plugin window.
The user after the error is more or less in a plateau should press the "Reset
weights" button and observe the error evolution. Afterwards, if necessary to finish
the demo the user can press the "Stop Application" button.
Used technology
The Migrating Desktop bases on the Java applet technology. It can be launched using
the Java Webstart technology or using a web browser with the appropriate Java Plug-
in
included in the Java Runtime Environment (JRE). We are basing on Swing libraries
for
designing graphical user interface, the Java CoG Kit version 1.2 is being used as
an interface to Globus (for operation on proxy and GridFTP/FTP) functionality and
Axis ver.1.1 web services client for communication with the Roaming Access
Server. Migrating Desktop follows OSGi Service Platform specification version 4
(August 2005) and is based on the same plugin engine as Eclipse platform. Currently
RAS for cooperation with EGEE infrastructure is using LCG2.6 platform but it is
foreseen to move to gLite.
