12-16 April 2010
Uppsala University
Europe/Stockholm timezone

A new middleware component for job monitoring in interactive mode: The SCoPE Experience

Apr 12, 2010, 6:18 PM
3m
Aula (Uppsala University)

Aula

Uppsala University

Poster Software services exploiting and/or extending grid middleware (gLite, ARC, UNICORE etc) Poster session

Speaker

Dr Silvio Pardi (INFN - Naples Unit)

Description

This work aims to extend the standard gLite middleware with a new functionality that enables the final users to interact with their submitted grid-jobs, for monitoring purposes. The new service is composed by a worker nodes Proxy server, with web-services interface VOMS compliant, easy to deploy and not invasive to respect the standard gLite solution. Through the Proxy, a generic user can connect, from the user interface, to the Worker Node that is taken his job. The tool has been implemented on the SCoPE infrastructure, to support the e-science community of the University of Naples Federico II

Impact

The impact of our work is very interesting in term of simplification of the interaction with the Grid, from the user point of view.
The proposed application, allows the final user to bypass the standard restrictions of a batch system, by using the standard gLite interaction. The user can control the effective state of his job and is enabled to control in realtime the job outputs, the memory occupation and many other parameters
The interactive monitor gives a very important instrument to the final user during the software development, by allowing to follow the flow of the jobs, and to do troubleshooting with a strong impact in term of time saving during the application tuning.
Finally the simplicity of the system allows the user to interact with his job in a transparent way, without any additional concerns on the use of computer grid.

Conclusions and Future Work

The interaction with the job during the running is a very important activity during the software preparation, job tuning and pre-production phase, and is a requirement of many scientific communities involved in computational science. In this work we proposed a solution, secure, simple to deploy and to use, and transparent with a minimum impact on the standard gLite middleware. Currently our implementation can work on the PBS queue system, and in the next months we plan to optimize the use of this application for parallel jobs, and also to port the software on different batch systems.

Detailed analysis

The application is based on a client-server infrastructure, with the core installed on the Proxy machine that works as server. The Proxy is composed of a web service interface using the https protocol, that pilots a set of local applications, responsible to find information about the user-jobs and to open and close GSI secure connections from the User Interface to the Worker Nodes.
The interaction with the user is done through a CLI, that take in input the Grid-Job ID of the process that we want to control. The client contacts the web-service via https and sends the JobID plus the use proxy certificate. The server verifies the user credential and the VOMS extensions, then interacts with the CE, through the gsiftp protocol, in order to understand which WN hosts the user-job and to find other information about the job. Finally the Proxy server opens a GSISSH secure connection with the User Interface and gives a shell on the WN that runs the user-job with the right local account, in the execution folder. The shell is closed by the Proxy as soon as the job will finish.
The adoption of this software, does not involve any alterations of the standard glite roles (i.e. CE or WN).

Keywords Interactive session, Grid Shell, monitor
URL for further information http://people.na.infn.it/spardi

Primary author

Dr Silvio Pardi (INFN - Naples Unit)

Co-authors

Dr Francesco Palmieri (University of Naples Federico II) Prof. Guido Russo (University of Naples Federico II) Dr Jonathan Cacace (University of Naples Federico II) Prof. Leonardo Merola (University of Naples Federico II)

Presentation materials

There are no materials yet.