EGEE User Forum

1 Mar 2006, 08:00 → 3 Mar 2006, 20:00 Europe/Zurich

CERN

The EGEE (Enabling Grids for E-sciencE) project provides the largest production grid infrastructure for applications. In the first two years of the project an increasing number of diverse users communities have been attracted by the possibilities offered by EGEE and have joined the initial user communities. The EGEE user community feels it is now appropriate to meet to share their experiences, and to set new targets for the future, including both the evolution of the existing applications and the development and deployment of new applications onto the EGEE infrastructure.

The EGEE Users Forum will provide an important opportunity for innovative applications to establish contacts with EGEE and with other user communities, to plan for the future usage of the EGEE grid infrastructure, to learn about the latest advances, and to discuss the future evolution in the grid middleware. The main goal is to create a dynamic user community, starting from the base of existing users, which can increase the effectiveness of the current EGEE applications and promote the fast and efficient uptake of grid technology by new disciplines. EGEE fosters pioneering usage of its infrastructure by encouraging collaboration between diverse scientific disciplines. It does this to evolve and to expand the services offered to the EGEE user community, maximising the scientific, technological and economical relevance of grid-based activities.

We would like to invite hands-on users of the EGEE Grid Infrastructure to Submit an Abstract for this event following the suggested template.

EGEE User Forum Web Page EGEE USer Forum Web Page

Wednesday, 1 March

13:00 → 14:00 Lunch

Thursday, 2 March

12:30 → 14:00 Lunch

14:00 → 18:35 2d: VO tools - Portals

14:00 Introduction

14:05 VO Support

14:10 Experience Supporting the Integration of LHC Experiments Software Framework with the LCG Middleware

The LHC experiments are currently preparing for data acquisition in 2007 and because of the large amount of required computing and storage resources, they decided to embrace the grid paradigm. The LHC Computing Project (LCG) provides and operates a computing infrastructure suitable for data handling, Monte Carlo production and analysis. While LCG offers a set of high level services, intended to be generic enough to accommodate the needs of different Virtual Organizations, the LHC experiments software framework and applications are very specific and focused on the computing and data models. The LCG Experiment Integration Support team works in close contact with the experiments, the middleware developers and the LCG certification and operations teams to integrate the underlying grid middleware with the experiment specific components. The strategical position between the experiments and the middleware suppliers allows EIS team to play a key role at communications level between the customers and the service providers. This activity is the source of many improvements on the middleware side, especially by channelling the experience and the requirements of the LHC experiments. The scope of the EIS activity encompasses several areas: 1) Understanding of the experiment needs 2) Identify open issues and possible solutions 3) Develop specific interfaces, services and components (when missing in or not yet satisfactory) 4) Provide operational support during Data Challenges, Service Challenges and massive productions. 5) Provide and maintain the user documentation; 6) Provide tutorial for the users community In the last year, the focus has been extended also to non High-Energy Physics communities like Biomed, GEANT4 and UNOSAT. In this work we discuss the EIS experience, describing the issues raising in the organization of the Virtual Organization support and the achievements, together with the lessons learned. This activity will continue in the framework of EGEE II, and we believe could be an example for several users communities on how to optimise their uptake of grid technology in the most efficient way.

Speaker: Dr roberto santinelli (CERN/IT/PSS)

Slides Slides

14:25 User and virtual organisation support in EGEE

User and virtual organisation support in EGEE Providing adequate user support in a grid environment is a very challenging task due to the distributed nature of the grid. The variety of users and the variety of Virtual Organizations (VO) with a wide range of applications in use add further to the challenge. The people asking for support are of various kinds. They can be generic grid beginners, users belonging to a given Virtual Organization and dealing with a specific set of applications, site administrators operating grid services and local computing infrastructures, grid monitoring operators who check the status of the grid and need to contact the specific site to report problems; to this list can be added network specialists and others. Wherever a user is located and whatever the problem experienced is, a user expects from a support infrastructure a given set of services. A non-exhaustive list is the following: a) a single access point for support; b) a portal with a well structured sources of information and updated documentation concerning the VO or the set of services involved; c) experts knowledgeable of the particular application in use and who can even discuss with the user to better understand what he/she is trying to achieve (hot- line); help integrating user applications with the grid middleware; d) correct, complete and responsive support; e) tools to help resolve problems (search engines, monitoring applications, resources status, etc.); f) examples, templates, specific distributions for software of interest; g) integrated interface with other Grid infrastructure support systems; h) connection with the grid developers and the deployment and operation teams; i) assistance during production use of the grid infrastructure. With the Global Grid User Support (GGUS) infrastructure, EGEE attempts to meet all of these expectations. The current use of the system and the user satisfaction ratings have shown that the goal has been achieved with a certain success for the moment. As of today GGUS has shown to be able to process up to 200 requests per day and provides all above listed services. In what follows we discuss the organization of the GGUS system, how it meets the users’ needs, and the current open issues. The model of the existing EGEE Global Grid User Support (GGUS) is as follows. The support model in EGEE can be captioned "regional support with central coordination". Users can submit a support request to the central GGUS service, or to their Regional Operations' Center (ROC) or to their Virtual Organisation (VO) helpdesks. Within GGUS there is an internal support structure for all support requests. The ROCs and VOs and the other project wide groups such as middleware groups (JRA), network groups (NA), service groups (SA) and other grid infrastructures (OSG, NorduGrid, etc.) are connected via a central integration platform provided by GGUS. GGUS central helpdesk also acts as a portal for all users who do not know where to send their requests. They can enter them directly into the GGUS system via a web form or e-mail. This central helpdesk keeps track of all service requests and assigns them to the appropriate support groups. In this way, formal communication between all support groups is possible. To enable this, each group has built an interface (e-mail and web front-end, or interface between ticketing systems) between its internal support structure and the central GGUS application. In the central GGUS system, first line support experts from the ROCs and the Virtual Organizations will do the initial problem analysis. Support is widely distributed. These experts are called Ticket Processing Managers (TPM) for generic first line support (generic TPM) and for VO specific first line support (VO TPM). These experts can either provide the solution to the problem reported or escalate it to more specialized support unit that provide network, middleware and grid service support. They may also refer it to specific ROCs or VO experts. Behind the specialized VO TPM support units, people belonging to EGEE/NA4 groups such as the Experiment Integration Support group (EIS) help VO users with on-line support and the integration of the VO specific applications with the grid middleware. Such people can also recognize if a problem is application specific and forward the problem to more VO specific support units connected to GGUS. TPM and VO TPMs have also the duty of following tickets, making sure that users receive an adequate answer, coordinating the effort of understanding the real nature of the problem and involving more than one second level support unit if needed. The following figure depicts the ticket flow. To provide appropriate user support, the distributed structure of EGEE and the VOs has to be taken into account. The community of supporters is therefore distributed. Their effort is coordinated centrally by GGUS and locally by the local ROC support infrastructures. The ROC provides adequate support to classify the problems and to resolve them if possible. Each ROC has named user support contacts who manage the support inside the ROC and who coordinate with the other ROCs’ support contacts. The classification at this level distinguishes between operational problems, configuration problems, violations of service agreements, problems that originate from the resource centres and problems that originate from global services or from internal problems in the software. Problems that are positively linked to a resource centre are then transferred to the responsibility of the ROC with which the RC is associated. MEETING USER NEEDS As explained above, GGUS provides therefore a single entry point for reporting problems and dealing with the grid. In collaboration with the EGEE EIS team, the EGEE User Information Group, NA3, and the entire EGEE infrastructure, GGUS offers a portal where users can find up-to-date documentation, and powerful search engines to find answers to resolved problems and examples. Common solutions are stored in the GGUS knowledge database and Wiki pages are compiled for frequent or undocumented problems/features. GGUS offers hot lines for users and supporters and a VRVS chat room to make the entire support infrastructure available on-line to users. Special tools and grid middleware distributions are made available by the NA4/EIS team for GGUS users. GGUS is interfaced with other grids’ support infrastructures such as in the case of OSG and NorduGrid. Also, GGUS is used for daily operations to monitor the grid and keep it healthy. Therefore, specific user problems can be directly communicated to the Grid Operation Centers and broadcasted to the entire grid community. GGUS is used also to follow and track down problems during stress testing activities such as the HEP experiments production data challenges and the service challenges. OPEN ISSUES Even-though GGUS has proven to provide useful services, there are still many things that need improvement. Concerning users and VOs, in particular, we have identified the following: Small VOs do not have the resources to implement their part of the model The large VOs such as the LHC experiments have people who provide support for the applications which the VO has to run as part of its work. These people are contacted by GGUS when tickets are assigned to the VO or then the problem needs immediate or on-line attention. It has proven difficult for some of the small VOs to provide such a service. In this case, GGUS still provides support for the VO, but if the problem is application related and cannot be resolved, then it has to be put into the state ‘unsolvable’. Supporters have other jobs to do In EGEE, almost everyone providing support does so as part of their job. It is not usually a major part of their job. Some times it is difficult to ensure responsiveness. There is a small team which maintains and develops the GGUS system. Supporters are concentrated in a few locations The resources of the grid are widely distributed over 180 locations, and there are people in all of these locations looking after the basic operation of the computers. However this is not the case for higher level support such as support for a VO application. This tends to exist in only a small number of locations, with a small number of supporters. Scalability is constrained by the availability of supporters The number of people who can provide support for basic operations is large, but the number of people who can provide support for higher level services is small. As the VOs become larger this will become a constraint to growth unless more supporters are found. Limited experience in handling a large number of tickets As part of the development of the GGUS system, it has been exercised by generating tickets. As the system is built from industry standard software parts using Remedy and Oracle, it has been found to be reliable. We believe however that if large numbers of tickets are submitted that it will show the limitations in the system. Limited engagement of existing VOs in the implementation of GGUS There is an organisation within EGEE called Executive Support Committee (ESC). The ESC has representatives from all of the ROCs of EGEE. This organisation meets once per month by telephone to discuss the operations and development of the support system and to decide on actions and priorities for the work. The present VOs have found it difficult to provide people for involvement with this work. CONCLUSION The GGUS system is now ready for duty. During 2006, it is expected that there will be a large number of tickets passing through the system as the LHC VOs move from preparing for service to being in production. It is also expected that the number of Virtual Organisations will grow as the work of EGEE-II proceeds. There will also be an increase in the number of support units involved with GGUS, and an increase in the number of ROCs and RCs. Acronyms EGEE Enabling Grids for E-sciencE EIS Experiment Integration Support GGUS Global Grid User Support HEP High Energy Physics JRA Joint Research Activity of EGEE LHC Large Hadron Collider NA Network Activity OSG Open Science Grid RC Resource Centre ROC Regional Operations' Centre SA Service Activity TPM Ticket Process Management VO Virtual Organisation VRVS Virtual Rooms Videoconferencing System Wiki Web technology for collaborative working

Speaker: Flavia Donno (CERN)

Slides Donno.pdf slides

14:45 Discussion

15:00 VO Portals

15:05 EnginFrame as FrameWork for Grid Enabled Web Portals on industrial and research contexts.

EnginFrame is a Web-based innovative technology, by the Italian company Nice S.r.l., that enables access and exploitation of Grid-enabled applications and infrastructures. It allows organizations to provide application oriented computing and data services to both users (via Web browsers) and in-house or ISV applications (via SOAP/WSDL based Web services), hiding all the complexity of the underlying Grid infrastructure. In particular, EnginFrame greatly simplifies the development of Web portals exposing computing services that can run on a broad range of different computational Grid systems (including Platform LSF, Sun Grid Engine, Altair PBS, Globus, LCG-2 and gLite grid middlewares by European project EGEE). EnginFrame supports several open and vendor neutral standards and seamlessly integrates with JSR168 compliant enterprise portals, distributed file systems, GUI virtualization tools and different kinds of authentication systems (including Globus GSI, MyProxy and a wide range of enterprise solutions). Because EnginFrame greatly simplifies the use of Grid-enabled applications and services, it has already been adopted by numerous important industrial companies all over the world, besides many leading research & educational institutes. Service publishing is achieved by developing simple XML-based descriptions of the interface and business logic representing the actual services implementation. EnginFrame receives incoming requests via standard Web protocols over HTTP, authenticates and authorizes the requests and then executes the required actions into the underlying Grid computational environment. Then, EnginFrame gathers the results and transforms them into a suitable format before sending the response to the client. Transformation of results is performed according to the nature of the client: HTML for Web browsers and XML for Web services client applications or RSS clients. For each submitted service, a data staging area (the "spooler") for the service input and output files is created on the file system. Most of the information managed by EnginFrame are described by dynamically generated XML documents. The source of such information is typically the service execution environment: an XML abstraction layer aims to submit service actions and translate raw results coming from the computational environment into XML structures. The XML abstraction layer is designed to decouple EnginFrame from the actual Grid working environment, hiding the specific Grid technology solution. This characteristic makes possible to easily extend EnginFrame functionalities by developing ad-hoc plugins for specific computational and data Grid middlewares. To support the integration of data Grid middleware solutions, EnginFrame introduces the concept of Virtual Spoolers that represent distributed data areas that reside outside the EnginFrame spoolers file system, but that can be remotely accessed by EnginFrame itself through the targeted data Grid technology. The structure and the content of a Virtual Spooler is described by a dynamically generated XML document. Thus, the access to data catalogs and storage technologies is provided in a very easy way and their contents can be inspected like a "browse a file". Concerning technical aspects, there are some key issues that must be addressed properly in Grid Portal development in industrial contexts: grid security and authentication aspects are critical both at Grid middleware-level and at access-level; the authorization system should be built into the Grid system, enabling a fine-grained access control to resources (datasets, licenses, computing resources); the accounting system, suitable to collect the resource usage and supporting reporting and billing services, should be able to collect the records from the various Grid nodes and merge them according to the business needs; application integration and deployment to the Grid context, as well as administration should be standardized and simplified; the access and the exploitation of Grid enabled applications by the end users should be simplified to the level of a web browsing experience; the users shouldn't need to be aware of the Grid infrastructure running the application, to perform their tasks. For the industrial/engineering companies, the long and complex process that goes from the design of an industrial product to manufacturing, involves the cooperation of dozens or hundreds of people, departments or companies, often SMEs, ranging from engineering service providers to component suppliers. This can be regarded as a “virtual organization”, made of individual members or groups of people from the various companies that share, with a well defined role and profile, the overall project goal, often composed of geographically distant members, which would benefit from increased, real-time sharing of information and IT infrastructures, while preserving the intellectual properties of each of the project members. There are a number of factors, ranging from human, to organizational, to technical and to business aspects that are only partially addressed by current GRID technologies, that pratically limit the adoption of this approach. The Web-centric approach lets users access any service virtually from anywhere, at any time, over any network and platform, including Personal Digital Assistant and Cellular Phones, thus supporting the ubiquitous access to the Grid. Built on the experience of Industrial and Engineering requirements, the EnginFrame system has been designed to enable addressing effectively the above mentioned values, while minimizing the efforts to build and maintain a successful Grid Portal solution. GENIUS Portal [1], based and powered by EnginFrame, jointly developed by INFN and NICE srl within the INFNGrid Project, allows in a very easy way the integration of applications ported to be executed on LCG-2 and gLite Middlewares, and many applications have been implemented on GILDA dissemination testbed [2] from the beginning and shown within dozens of tutorials, giving to the user an easy way to run jobs on the grid and to manage own data using the virtualizations offered by exposed services at different levels, locally, remotely, on catalogs. On the other hand, using the EnginFrame Framework, GENIUS Portal has inherited all the features, deriving from years of development and experience into industrial contexts, like scalability, flexibility, easy maintenance, security, fault tolerance, connectivity, data management, authorization, usability. Conclusions. The adoption of this innovative technology has given industries and engineering companies very important benefits in improvements in productivity running on Grid-enabled infrastructures. GENIUS, by staying aligned with the middleware development, can be an instrument to facilitate a dialog between research and industrial contexts based on a high-level services approach. This dialog can give also a very high added-value for both worlds, to spread the use of Grid infrastructures and generate a critical mass of awareness and trust. References. [1] "GENIUS: a simple and easy way to access computational and data grids" G. Andronico, R. Barbera, A. Falzone, P. Kunszt, G. Lo Rè, A. Pulvirenti, A. Rodolico - Future Generation of Computer Systems, vol. 19, no. 6 (2003), 805-813. [2] "GILDA: The Grid INFN Virtual Laboratory for Dissemination Activities" G. Andronico, V. Ardizzone, R. Barbera, R. Catania, A. Carrieri, A. Falzone, E. Giorgio, G. La Rocca, S. Monforte, M. Pappalardo, G. Passaro, G. Platania - TRIDENTCOM 2005: 304-305.

Speakers: Alberto Falzone (NICE srl), Andrea Rodolico (NICE srl)

Slides slides

15:20 Discussion

15:30 VO Monitoring

15:35 GridICE monitoring for the EGEE infrastructure

Grid computing is concerned with the virtualization, integration and management of services and resources in a distributed, heterogeneous environment that supports collections of users and resources across traditional administrative and organizational domains. One aspect of particular importance is Grid monitoring, that is the activity of measuring significant Grid resource-related parameters in order to analyze usage, behavior and performance of a Grid system. The monitoring activity can also help in the detection of fault situations, contract violations and user-defined events. In the framework of the EGEE (Enabling Grid for E-sciencE) project, the Grid monitoring system called GridICE has been consolidated and extended in its functionalities in order to meet requirements from three main categories of users: Grid operators, site administrators and Virtual Organization (VO) managers. Besides the specific needs of these categories, GridICE offers a common sensing, collection and presentation framework enabling to share common features, while also offering user-specific needs. A first common aspect to the different users is the set of measurements to be performed. Typically, there is a wide number of base measurements that are of interest for all parties, while a small number is specific to them. What makes the difference is the aggregation criteria required to present the monitoring information. This aspect is intrinsic to the multidimensional nature of monitoring data. Example of aggregation dimensions identified in GridICE are: the physical dimension referring to geographical location of resources, the Virtual Organization (VO) dimension, the time dimension and the resource identifier dimension. As an example, considering the entity 'host' and the measure 'number of started processes in down state', the Grid operator can be interested in accessing the sum of the measurement values for all the core machines (e.g., workload manager, computing element, storage element) in the whole infrastructure, while the Virtual Organization manager can be interested in the sum of the measurement values for all the core machines that are authorized to the VO members. Finally, the site administrator can be interested in accessing the sum of the measurement values for all machines part of its site. Another aspect that is common to all the consumers is being able to start from summary views and to drill down to details. This feature can enable to verify the composition of virtual pools or to sketch the sources of problems. As regards the distribution of monitoring data, GridICE follows a 2-level hierarchical model: the intra-site level is within the domain of an administrative site and aims at collecting the monitoring data at a single logical repository; the inter-site level is across sites and enables the Grid-wide access to the site repository. The former is typically performed by a fabric monitoring service, while the latter is performed via the Grid Information Service. In this sense, the two levels are totally decoupled and different fabric monitoring services can be adapted to publish monitoring data to GridICE, thought the proposed default solution is the CERN Lemon tool. Considering the sensing activity, GridICE adopts the whole set of measures defined in the GLUE Schema 1.2, further it provides extensions to cover new requirements. The extensions include a more complete host-level characterization, Grid jobs related attributes and summary info for batch systems (e.g., number of total slots, number of worker nodes that are down). The development activity in the EGEE project has focused on the following aspects: the redesign of the presentation level took into consideration the usability principles and compliance with W3C standards; sensors for measuring parameters related to Grid job have been re-engineered to scale to the number of jobs envisioned by big sites (e.g., LCG Tier 1 centers); new sensors have been written to deal with summary information for computing farms; stability and reliability of both server and sensors. The deployment activity covers the whole EGEE framework with several server instances supporting the work of different Grid sub-domains (e.g., whole EGEE Grid domain, ROC domain, national domain). Other Grid projects have adopted GridICE for monitoring their resources (e.g., EUMedGrid, EUChinaGRID, EELA). As regards the user experience, GridICE has proven to be useful to different users in different ways. For instance, Grid operators have summary views for aspects such as information sources status and host status. Site administrators appreciate the job monitoring capability showing the status and computing activity of the jobs accepted in the managed resources. VO managers use GridICE to verify the available resources and their status before to start the submission of a huge number of jobs. Finally, GridICE has been positively adopted in dissemination activities. While GridICE has reached a good maturity level in the EGEE project, many challenges are still open in the dynamic area of Grid systems. The short term plans are: (1) as regards the discovery process, there is the need to finalize the transition from the MDS-based information service to the gLite service discovery plus publisher services such as R-GMA producers and CEMon; (2) integration with information present in the Grid Operation Center (GOC) database for accessing resource planned downtime and other management information; (3) tailored sensors for the workload management service; (4) sensors for measuring data transfer activities across Grid sites. References: Dissemination website: http://grid.infn.it/gridice Publications: http://grid.infn.it/gridice/index.php/Research/Publications

Speaker: Mr Sergio Andreozzi (INFN-CNAF)

Slides Slides

15:50 Discussion

16:00 Coffee break

16:30 VO Software Management

16:35 Supporting legacy code applications on EGEE VOs by GEMLCA and the P-GRADE portal

Grid environments require special grid-enabled applications capable of utilising the underlying middleware services and infrastructures. Most Grid projects so far have either developed new applications from scratch, or significantly re-engineered existing ones in order to be run on their platforms. This practice is appropriate only in the context where the applications are mainly aimed at proving the concept of the underlying architecture. However, as Grids become stable and commonplace in both scientific and industrial settings, a demand will be created for porting a vast legacy of applications onto the new platform. Companies and institutions can ill afford to throw such applications away for the sake of a new technology, and there is a clear business imperative for them to be migrated onto the Grid with the least possible effort and cost. Grid computing has reached the point where reliable infrastructures and core Grid services are available for various scientific communities. However, not even the EGEE Grid contains any tool to support the turning of legacy applications into Grid services that provide complex functions on top of the core Grid layer. The Grid Execution Management for Legacy Code Architecture (GEMLCA), presented in this paper, enables legacy code programs written in any source language (Fortran, C, Java, etc.) to be easily deployed on the EGEE Grid as a Grid service without significant user effort. GEMLCA does not require any modification of, or even access to, the original source code. A user-level understanding, describing the necessary input and output parameters and environmental values – such as the number of processors or the job manager required – is all that is needed to port the legacy application binary onto the Grid. Moreover, since GEMLCA has been integrated with the P-GRADE Portal, end-users can publish legacy applications as Grid services and can invoke legacy code services as a special kind of job (node) inside their workflows by an easy to use graphical portal interface. The GEMLCA - P-GRADE Portal has been operating for the UK NGS community as a service since September 2005. Recently, the researchers of the University of Westminster and MTA SZTAKI have developed the EGEE-specific version of this tool. The EGEE-specific GEMLCA P-GRADE Portal offers the same legacy code management and workflow-oriented application development and execution facilities for EGEE research communities that have been provided on the UK NGS for more than six months now. On top of the JSR-168 compliant portlets of the P-GRADE Portal (credential management, workflow enactment, etc) the GEMLCA-specific version contains an additional portlet that can be used to turn legacy applications into Grid services and to offer these services to other users of the portal. These users can invoke the legacy code services with their own custom input data, moreover, they can integrate legacy code services with newly developed codes inside their workflows. The portal environment contains a GEMLCA-specific editor to help users define such workflows. The workflow enactment service integrated into the Portal is capable to forward job submission and legacy code service invocation requests to appropriate providers. While the core EGEE sites are responsible for job execution, the “legacy code repository” component of the portal server handles legacy code invocation requests. This centralised repository provides opportunity for portal users to share applications with each other. The facility is a natural step to extend the concept of Virtual Organizations (VO). While the storage services of the EGEE Grid provide storage space for VO members in order to share data with each other, the code repository component of the GEMLCA P-GRADE Portal provides facility for VO members to share applications with each other. Moreover, since the P-GRADE Portal can be connected to multiple VOs at the same time, application sharing among the members of different VOs can take place through the Portal. According to the current notion of EGEE the Grid is separated into research domain specific VOs, each of them containing relatively small number of resources. This concept simply prohibits two scientists working on two different scientific domains to collaborate with each other. Because these researchers are members of two different VOs there is no way for them to share applications with each other. However, by publishing their applications in the “legacy code repository” component of the GEMLCA P-GRADE Portal they can share these codes with other members of the whole EGEE community. This facility paves the way for revolutionary results in interdisciplinary research. Besides the GEMLCA P-GRADE Portal the presentation will introduce an urban traffic simulation application developed on the EGEE Grid using this tool. The traffic simulation is based on a workflow consisting of three types of components. The Manhattan legacy code (component 1) is an application to generate inputs for the MadCity simulator: a road network file and a turn file. The MadCity road network file is a sequence of numbers, representing a road topology of a road network. The MadCity turn file describes the junction manoeuvres available in a given road network. Traffic light details are also included in this file. MadCity (component 2) is a discrete-time microscopic traffic simulator that simulates traffic on a road network at the level of individual vehicles behaviour on roads and at junctions. After completing the simulation, a macroscopic trace file, representing the total dynamic behaviour of vehicles throughout the simulation run, is created. Finally a traffic density analyser (component 3) compares the traffic congestion of several runs of the simulator on a given network, with different initial road traffic conditions specified as input parameters. The component presents the results of the analysis graphically. The lecture will use this application to describe how portal users can integrate their domain-specific applications into a large distributed program to solve the complex problem of traffic simulation. This example will present the benefits of portal-based collaborative work on the EGEE.

Speaker: Mr Gergely Sipos (MTA SZTAKI)

Slides slides

16:50 ETICS: eInfrastructure for Testing, Integration and Configuration of Software

A broad range of projects from a spectrum of disciplines involve the development of software born from the collaborative efforts of partners from geographically spread locations. Such software is often the product of large-scale initiatives as new technological models like the Grid are developed and new e-Infrastructures are deployed to help solve complex, computational-intensive problems. Recent experience in such projects has shown that the software products often risk suffering from lack of coherence and quality. Among the causes of this problem we find the large variety of tools, languages, platforms, processes and working habits employed by the partners of the projects. In addition, the issue of available funding for maintenance and support of software after the initial development phase in typical research projects often prevents the developed software tools from reaching production-level quality. Establishing a dedicated build and test infrastructure for each new project is inefficient, costly and time-consuming and requires specialized resources, both human and material, that are not easily found. The ETICS effort aims to support such research and development initiatives by integrating existing procedures, tools and resources in a coherent infrastructure, additionally providing an intuitive access point through a web portal and a professionally managed, multiplatform capability based on Grid technologies. The outcome of the project will be a facility operated by experts that will enabled distributed research projects to integrate their code, libraries and application, validate the code against standard guidelines, run extensive automated tests and benchmarks, produce reports and improve the overall quality and interoperability of the software. ETICS objectives are not to develop new software but to adapt and integrate already existing capabilities, mainly open source, providing other research project with the possibility to focus their effort in their specific research field and to avoid wasting time and resources in such, required, but expensive, activity. Throughout the duration of the project the ETICS partners will investigate the advantages of making use of the ETICS services, the technical challenges relates to running such a facility and its sustainability for the future. The vision and mission of ETICS will be accomplished through the following objectives: • Establish an international and well managed capability for software configuration, integration, testing and benchmarking for the scientific community. Software development projects will use the capabilities provided by ETICS to build and integrate their software and perform complex distributed test and validation tasks • Deploy and if necessary adapt best-of-breed software engineering tools and support infrastructures developed by other projects (EGEE, LCG, NMI) and other open- source or industrial entities and organize them in a coherent, easy-to-use set of on-line tools • Create a repository of libraries that project can readily link against to validate their software in different configurations conditions • Leverage a distributed infrastructure of compute and storage resource to support the software integration and testing activities of a broad range of software development efforts. • Collect, organize and publish middleware and applications configuration information to facilitate interoperability analysis at the early stages of development and implementation • Collect from the scientific community sets of test suites that users can apply to validate deployed middleware and applications and conversely software providers can use to validate their products for specific uses • Raise awareness of the need for high-quality standards in the production of software and promote the identification of common quality guidelines and principles and their application to software production in open-source academic and research organization. Study the feasibility of a “Quality Certification” for software produced by research projects • Promote the international collaboration between research projects and establish a virtual community in the field of software engineering contributing to the development of standards and advancement in the art From the perspective of Grid application developers, the ETICS service should provide them with the means to automate their build and test procedures. In the longer term, via the ETICS service, users will be able to explore meaningful metrics pertaining to the quality of their software. Further, as Grid application level services (most concerned by providers of Grid turn key solutions), the ETICS service will also offer a repository or already built components, services and plug- ins, with a published quality level. Furthermore, the quality metrics provided by the ETICS services and available for each artifact in the repository will help guiding the user in selecting reliable software dependencies. Finally, the repository will also contain pre-build artifacts for specific hardware platforms and operating systems, which will help the developers to assess the platform independence of their entire service, including each and every dependency the service is relying on. In conclusion, most Grid and distributed software project invest in a build and test system in order to automatically build and test their software and monitor key quality indicators. ETICS takes requirements from many Grid and distributed projects and with the help of Grid middleware, offers a generic yet powerful solution for building and testing software. Finally, building software via such a systematic can provide a rich pool of published quality components, services and plug-ins, on which the next generation of Grid and distributed applications could be based on and composed of.

Slides MEB.pdf MEB.ppt

17:05 Discussion

17:15 Other Tools and Infrastructures

17:30 Universal Acessibility to the Grid via Metagrid Infrastructure

This paper discusses the concept of universal accessibility [1, 2] to the grid within the context of selected application domains involving social interaction such as e-hospital, collaborative engineering, enterprise, e-government, and the media. Based on this discussion the paper proposes a metagrid infrastructure [3] as an approach to provide universal accessibility to the grid. Universal accessibility is rooted in the concept of Design for All in Human Computer Interaction[1, 2]. It aims at efficiently and effectively addressing the numerous and diverse accessibility problems in human interaction with software applications and telematic services. So far, the key concept of universal accessibility has been supported by various development methodologies and platforms [4, 5]. Various application domains benefited from research and development in this area, including among others interactive television and media [6, 7]. Porting the concept of universal accessibility to the grid is faced by major obstacles attributed to the following: (a) the lack of an underlying functionality similar to that of a desktop operating system allowing the plug and play of resources and the direct user interaction with these resources; (b) the dilemma between hiding the grid versus making it more transparent; and (c) the software engineering practice adopted in grid middleware development, where the bottom up approach that is predominant [8] conflicts with the ethos of universal accessibility that considers accessibility at design time. These obstacles and their impacts on universal accessibility to the grid are discussed with reference to four application domains including collaborative applications such as e-hospital, collaborative engineering, enterprise applications, the media, and e-government. In collaborative applications the key obstacle for universal accessibility to the grid is provision of interactivity while respecting various Service Level Agreements (SLAs). Several efforts are underway to resolve this issue [9, 21], but no versatile solutions have emerged so far. In the enterprise the major concern is the management of an integrated data centre [10]; the key obstacle confronted is that while already offering data-intensive computational power the grid is quite immature in its provision of permanent storage of data. This is very much a live issue in grid middleware development. In the media the major challenge is the direct access to remote external devices at the grid boundaries. For e-government accommodating various forms of interaction [11], such as government-to-government (G2G), government-to-citizen (G2C), and government-to-business (G2B), is paramount, whilst devoting a major focus on data semantics, not just structure. So far universal accessibility to the grid was addressed from various perspectives. Efforts undertaken involved: (a) the development of grid middleware supporting interaction with heterogeneous mobile devices [12, 13]; (b) the use of operating system mobility for configuring grid application on a PC and then migrating the entire application together with the operating system instance onto the grid [14]; (c) the development of a shopping cart system based on the Web Service Resource Framework WSRF [15]; (d) the design of an approach for middleware development, based on wrapping the computational and resource intensive tasks, to allow the accessibility to the grid via hand held devices [16, 22]; (e) the development of common web-based grid application portals allowing the applications' users to customize their interfaces to the grid [17, 23, 24]; (f) the development of application models for the grid [18]; and (g) addressing security issues raised by granting grid accessibility via various media delivery channels (such as wireless devices) [19]. While each of these efforts towards universal accessibility to the grid does address the problem to some extent, none of them enables a complete solution. This paper proposes an approach, based on a metagrid infrastructure, that can potentially host solutions to all issues related to universal accessibility to the grid. This metagrid infrastructure was used thus far in the context of grid interoperability [3]. Our proposed approach extends the notion of interoperability to embrace grid application interoperability (interactivity and universal accessibility). While heavily based on existing grid middleware services and architecture such as EGEE, Globus, CrossGrid, GridPP and GGF [25, 26, 23, 27, 28], the metagrid infrastructure hosts one or more target grid techologies (e.g. it has been demonstrated simultaneously hosting WebCom, LCG2 and GT4) while also supporting its own services that provide things like universal accessibility that the target grid technologies do not. By doing so it firmly places the user within the metagrid environment rather than in any one target grid environment. The user obtains universal accessibility via the metagrid services, and the target grid technologies are relieved of the need to support direct user and device interactions. By way of example, services currently offered by the metagrid infrastructure include a transparent grid filesystem [26] that supplies a vital missing component beneath existing middleware. The grid filesystem can support universal accessibility by supporting all forms of data access (r/w/x) in the course of collaborative interaction (collaborative engineering and e-hospital), by providing a logical user view of grid data (to support integration of the data centre in the enterprise), and by helping locate (discover) data in the course of interaction in media applications. In so doing it can improve the utility of, for example, the EGEE middleware. As further examples, proposed future services include special purpose discovery services to support various forms of interaction especially in media applications; and intelligent interpreters to support e-Government data semantics. The paper is divided in five sections. The first section introduces the concept of universal accessibility and its relevance to the grid. The second section discusses existing obstacles facing universal accessibility to the grid in application domains involving social interaction. The third section overviews existing efforts towards universal accessibility to the grid. The fourth section propose an approach for universal accessibility to the grid based on a metagrid infrastructure and prototype services offered by this infrastructure. The paper concludes with a summary and a future research agenda. = REFERENCES = [1]:: Stephanidis, D. Akoumianakis, M. Sfyrakis, and A. Paramythis, Universal accessibility in HCI: Process-oriented design guidelines and tool requirements, Proceedings of the 4th ERCIM Workshop on User Interfaces for All, Edited by Constantine Stephanidis, ICS-FORTH, and Annika Waern, SICS, Stockholm, Sweden, 19-21 October 1998 [2]:: Stephandis, C., From User interfaces for all to an information society for all: Recent achievements and future challenges, Proceedings of the 6th ERCIM Workshop User Interfaces for All, October 2000, Italy [3]:: Pierantoni, G. and Lyttleton, O. and O'Callaghan, D. and Quigley, G. and Kenny, E. and Coghlan, B., Multi-Grid and Multi-VO Job Submission based on a Unified Computational Model, Cracow Grid Workshop (CGW'05)Cracow, Poland, November 2005 [4]:: Stephanidis, C., Savidis, A., and Akoumianakis, D., Tutorial on Unified Interface Development: Tools for Constructing Accessible and Usable User Interfaces. Tutorial no. 13 in the 17th International Conference on Human Computer Interaction (HCI International'97), San Fransico, USA, 24-29 August. [Online] Available: http://www.ics.forth.gr/proj/at_hci/html/tutorials.htm [5]:: Akoumianakis, D., Stephanidis, C., USE-IT : A Tool for Lexical Design Assistance. In C. Stephanidis (ed.) User Interfaces for All Concepts, Methods and Tools. Mahwah, NJ: 9. Beynon, [6]:: Soha Maad, Universal Access For Multimodal ITV Content: Challenges and Prospects, Universal Access. Theoretical Perspectives, Practice, and Experience: 7th ERCIM International Workshop on User Interfaces for All, Paris, France, October 24-25, 2002. Revised Papers, N. Carbonell, C. Stephanidis (Eds.), Lecture Notes in Computer Science, Springer-Verlag Heidelberg, ISSN: 0302-9743, Volume 2615 / 2003, January 2003, pp.195-208. [7]:: Soha Maad, Samir Garbaya, Saida Bouakaz , From Virtual to Augmented Reality in Finance: A CYBERII Application, to appear in the Journal of Enterprise Information Management [8]:: S. Maad, B. Coghlan, G. Pierantoni, E. Kenny, J. Ryan, R. Watson, Adapting the Development Model of the Grid Anatomy to meet the needs of various Application Domains, Cracow Grid Workshop (CGW'05), Cracow, Poland, November, 2005. [9]:: Herbert Rosmanith, Dieter Kranzlmuller, glogin - A Multifunctional, Interactive Tunnel into the Grid, pp.266-272, Fifth IEEE/ACM International Workshop on Grid Computing (GRID'04), 2004. [10]:: Soha Maad, Brian Coghlan, Eamonn Kenny, Gabriel Pierantoni, The Grid For the Enterprise: Bridging Theory and Practice, paper in progress, Computer Architecture Group, Trinity College Dublin. [11]:: Maad S., Coghlan B., John R., Eamonn K., Watson R., and Pierantoni G. 2005, The Horizon of the Grid For E-Government, Proceeding eGovernment'05 Workshop, Brunel, United Kingdom, September 2005. [12]:: Hassan Jameel, Umar Kalim, Ali Sajjad, Sungyoung Lee, Taewoong Jeon, Mobile-to-Grid Middleware: Bridging the Gap Between Mobile and Grid Environments, Advances in Grid Computing - EGC 2005, European Grid Conference, Amsterdam, The Netherlands, February 14-16, 2005, Editors: Peter M. A. Sloot, Alfons G. Hoekstra, Thierry Priol, Alexander Reinefeld, Marian Bubak, ISBN: 3-540-26918-5, Lecture Notes in Computer Science, Springer-Verlag GmbH, Volume 3470 / 2005, page 932. [13]:: Ali Sajjad, Hassan Jameel, Umar Kalim, Young-Koo Lee, Sungyoung Lee, A Component-based Architecture for an Autonomic Middleware Enabling Mobile Access to Grid Infrastructure, Lecture Notes in Computer Science, Springer-Verlag GmbH, Volume 3823/2005, pages 1225 - 1234. [14]:: Jacob Gorm Hansen, Eric Jul, Optimizing Grid Application Setup Using Operating System Mobility, Advances in Grid Computing - EGC 2005, European Grid Conference, Amsterdam, The Netherlands, February 14-16, 2005, Editors: Peter M. A. Sloot, Alfons G. Hoekstra, Thierry Priol, Alexander Reinefeld, Marian Bubak, ISBN: 3-540-26918-5, Lecture Notes in Computer Science, Springer-Verlag GmbH, Volume 3470 / 2005, page 952. [15]:: Maozhen Li,Man Qi, Masoud Rozati, and Bin Yu, A WSRF Based Shopping Cart System, Advances in Grid Computing - EGC 2005, European Grid Conference, Amsterdam, The Netherlands, February 14-16, 2005, Editors: Peter M. A. Sloot, Alfons G. Hoekstra, Thierry Priol, Alexander Reinefeld, Marian Bubak, ISBN: 3-540-26918-5, Lecture Notes in Computer Science, Springer-Verlag GmbH, Volume 3470 / 2005, page 993. [16]:: Saad Liaquat Kiani, Maria Riaz, Sungyoung Lee, Taewoong Jeon, Hagbae Kim, Grid Access Middleware for Handheld Devices, Advances in Grid Computing - EGC 2005, European Grid Conference, Amsterdam, The Netherlands, February 14-16, 2005, Editors: Peter M. A. Sloot, Alfons G. Hoekstra, Thierry Priol, Alexander Reinefeld, Marian Bubak, ISBN: 3-540-26918-5, Lecture Notes in Computer Science, Springer-Verlag GmbH, Volume 3470 / 2005, page 1002. [17]:: Jonas Lindemann, Goran Sandberg, An Extendable GRID Application Portal, Advances in Grid Computing - EGC 2005, European Grid Conference, Amsterdam, The Netherlands, February 14-16, 2005, Editors: Peter M. A. Sloot, Alfons G. Hoekstra, Thierry Priol, Alexander Reinefeld, Marian Bubak, ISBN: 3-540-26918-5, Lecture Notes in Computer Science, Springer-Verlag GmbH, Volume 3470 / 2005, page 1012. [18}:: Fei Wu, K.W. Ng, A Loosely Coupled Application Model for Grids, Advances in Grid Computing - EGC 2005, European Grid Conference, Amsterdam, The Netherlands, February 14-16, 2005, Editors: Peter M. A. Sloot, Alfons G. Hoekstra, Thierry Priol, Alexander Reinefeld, Marian Bubak , ISBN: 3-540-26918-5, Lecture Notes in Computer Science, Springer-Verlag GmbH, Volume 3470 / 2005, page 1056 [19]:: Syed Naqvi, Michel Riguidel, Threat Model for Grid Security Services, Advances in Grid Computing - EGC 2005, European Grid Conference, Amsterdam, The Netherlands, February 14-16, 2005, Editors: Peter M. A. Sloot, Alfons G. Hoekstra, Thierry Priol, Alexander Reinefeld, Marian Bubak , ISBN: 3-540-26918-5, Lecture Notes in Computer Science, Springer-Verlag GmbH, Volume 3470 / 2005, page 1048 [20]:: Soha Maad, Brian Coghlan, Geoff Quigley, John Ryan, Eamonn Kenny, David O'Callaghan, Towards a Complete Grid Filesystem Functionality, submitted to special issue on Data Analysis, Access and Management on Grids, CALL FOR PAPERS , Future Generation Computer Systems, The International Journal of Grid Computing: Theory, Methods and Applications, Elsevier. [21]:: EU FP6 Project 031857: int.eu.grid, to start May, 2006. [22]:: Genius Portal, https://genius.ct.infn.it/ [23]:: Marian Bubak, Michal Turala, CrossGrid and Its Relatives in Europe, Proc.9th European PVM/MPI Users Group Meeting, LNCS, pp.14-15, Vol.2474, ISBN: 3-540-44296-0, Springer-Verlag, 2002. [24]:: M.Kupczyk, R.Lichwala, N.Meyer, B.Palak, M.Plociennik, P.Wolniewicz, Applications on Demand as the exploitation of the Migrating Desktop, Future Generation Computer Systems, pp.37-44, Vol.21, Issue 1, ISSN: 0167-739X, January 2005. [25]:: EU FP6 Project: Enabling Grids For E-sciencE, http://www.eu-egee.org/ [26]:: Globus Project, http://globus.org [27]:: GridPP Project, http://www.gridpp.ac.uk/ [28]:: Global Grid Forum (GGF), http://www.ggf.org/

Speaker: Dr Soha Maad (Trinity College Dublin)

Slides Slides

17:45 Methodology for Virtual Organization Design and Management

Introduction Contemporary grid environment achieved high level of maturity. With still increasing number of various available resources, their optimal exploitation becomes a significant problem. One of solutions to the problem are Virtual Organizations (VO), which groups users and resources to solve a particular problem or a set of problems. Each problem has its own specific requirements in name of computational power, network bandwidth, storage capacity, resource availability etc. During VO design process, appropriate resources have to be selected from all available. This task can be vary difficult or time consuming, if done manually. Current EGEE middleware (lcg 2.6 or glite 1.4.1) with VOMS or VOMRS systems address the problem of users management in existing VOs, offering web based interfaces for user registration and membership administration. However, creation of new VO is a heavy weight task, which is not automated. Existing EGEE procedures covers very well all administrative aspects, but in current form they are not feasible for automation of the VO creation task. There is no tool, which support design of new VO in EGEE environment. In the presentation we propose a methodology of VO design. This methodology can be used to build a knowledge based system, which would support the process of VO creation by automating tasks, which do not need user interaction and support user, when the interaction is necessary. The methodology is general and can be adapted to EGEE grid environment. The knowledge based system can be used to support design of new VO without changing existing EGEE procedures. Methodology We propose the way of VO design which consists of three steps: definition of the VO, creation of abstract VO, creation of solid VO. The first step of VO design is definition of the VO purpose with all requirements and constraints. This step has to be performed by an expert who knows the problem for which the VO is created. The definition of VO should be written in a form, which can be easily processed by machine, therefore we propose to use ontology for this task. The expert from the VO domain, does not have to be familiar with any ontology language. There is a need for a tool which will allow VO definition by fulfilling forms and questions. This tool can support the expert in the task, by providing hints and possible answers to questions. The next step is creation of abstract VO. Abstract VO consists of resource types and their amount which is needed to fulfill VO requirements. Abstract VO is derived from VO definition (and available resources). Abstract VO has exact information about required computational resources, storage resources and all other specific resources, like data sources (e.g. physical experiment), but does not aim to any specific instance of resource (site). However, the expert can state, that a specific site is required in VO, and this requirement will be fulfilled in the next step - creation of solid VO. For each resource type, there are functional and not functional requirements. The functional requirements are for example installed specific software on computational resources. Non functional requirements can be availability of resource or cost of usage. The last step of VO design is creation of solid VO. During this step abstract resources are exchanged by real instances. This task can be performed automatically. Resources selection is based on specified requirements and knowledge about the grid environment. The knowledge consists of many kinds of facts and information about each resource, like computational power, storage capacity, bandwidth (network, storage), statistics about resource availability, etc. Because of a dynamic nature of the Grid, available resources can change in time. To support VO requirements, unavailable resources should be replaced with new ones during the VO lifetime. Therefore the last step of VO design should be repeated any time when needed. During the first step of design, apart form getting the information on needed resources, a workflow, which defines the problem would be created. The workflow visualizes a process of VO usage, from data gathering, through each necessary step, like preprocessing, computations, postprocessing and visualisation. Using the workflow, one can easily generate a specific job description (can take advantage of DAG jobs) to solve the problem. This step can be done automatically. Summary Optimal resource utilization is a very important task for contemporary grid environments. With grid environments growth in size and complexity, this task becomes more and more complicated. We proposed the methodology, which can positively influence the process of optimal resource utilization by supporting design of a VO. Well designed VO hides size and complexity of the grid environments, reveling only parts, which are important for the specific problem (for which VO was created). Selection of appropriate resources for VO is time consuming task, therefore it's automation can significantly improve process of VO establishment. References [1] EGEE Home page <http://public.eu-egee.org/> [2] EGEE NA4 Home page <http://egee-na4.ct.infn.it/egaap/> [3] InteliGrid <http://www.inteligrid.com/> [4] KWf-Grid <http://www.kwfgrid.net/main.asp>

Speaker: Mr Lukasz Skital (ACC Cyfronet AGH / University of Science and Technology)

Slides Slides

18:00 Discussion

18:20 Wrap-up and Conclusions

Friday, 3 March

13:00 → 14:00 Lunch