WLCG Service Reliability Workshop

26 Nov 2007, 08:00 → 30 Nov 2007, 18:30 Europe/Zurich

IT Auditorium (CERN)

NEW! - VRVS has been booked as follows:

• Monday - Island
• Tuesday - Island
• Wednesday - Saturn
• Thursday - Plane
• Friday - Car

Tentative list of topics:

• Critical Services - Experiments' Viewpoint
• Reliability by design - follow-up on issues from WLCG Collaboration workshop in Victoria / CHEP
• Monitoring & end-to-end Service Reliability
• Middleware development - tips & techniques related to reliable by design.
  (Hopefully including a session on developing DB apps).
• WLCG Medium Term Requirements for Operations & Support

Target attendance: 30-50(?) people

Make your suggestions here

mailing list

Monday, 26 November

09:00 → 18:00 Critical services - Requirements

VRVS details: Island

09:00 Introduction and Idea of the Workshop

N.B. workshop summary will be given at Overview Board and GDB 1st week of December (next week)

Slides

• wlcg-ob-october.pdf
• wlcg-ob-october.ppt
• workshop-intro.pdf
• workshop-intro.ppt

09:20 Critical services - Requirements of the Experiments

10:20 coffee break

10:40 Techniques for implementing & running robust and reliable services

12:00 lunch break

13:30 Case Studies - WLCG Services (part 1)

15:00 coffee break

15:30 Case Studies - WLCG Services (part 2)

16:00 Case Studies - Experiment Services

FIO best practices wiki page

Speaker: Veronique Lefebure

Slides Critical_Services_new.pdf Critical_Services_new.ppt

Tuesday, 27 November

09:00 → 19:00 WLCG Operations - What is Required to support LHC experiments?

VRVS details: Island

09:00 Inter-site Problem Determination

Slides http://indico.cern.ch/materialDisplay.py?contribId=8&materialId=slides&confId=3578

09:40 FTS transfers - debugging tools

• Prototype tools and procedures on T0-export - Alexander Uzhinskiy
• Prototype tools and procedures at SARA - Ron Trompert
• Prototype tools and procedures at IN2P3 - David Bouvet
• Plans and direction (discussion) - Gavin McCance

Slides

• FTSMonitor_at_IN2P3-CC.pdf
• FTSMonitor_at_IN2P3-CC.ppt
• ftsmonitoring_nov2007.pdf
• ftsmonitoring_nov2007.ppt
• FTS_monitoring.pdf
• FTS_monitoring.ppt
• Monitoring_directions.pdf
• Monitoring_directions.ppt

10:40 coffee break

11:10 Mind the Gap

What can we do to prevent cracks opening (or widening) in the services?

Specific examples from recent times (i.e. during EGEE '07) include:

• SAM unavailability
• GridView - change of availability algorithm
• LFC - affected by Oracle client bug in 'old' versions

More communication and better planning would likely help. How (concretely) do we fix these problems before the deluge of data arrives?

11:40 Requirements for WLCG management dashboards

Speaker: John Shade (CERN)

Slides EGEE_Management_Dashboard.pdf EGEE_Management_Dashboard.ppt

12:20 lunch break

13:30 WLCG / EGEE / OSG operations and evolution in the coming years

WLCG / EGEE / OSG operations are now well established, through:

• Weekly joint operations meetings
• Bi-annual (roughly) workshops
• Sessions at WLCG collaboration workshops
• A set of tools, procedures and documentation.

This session summarises the current state of global operations as well as the possible evolution beyond (or even during) the next phase of EGEE (III) - in the light of the EGI Design Study.

In particular, we need to establish a clear view of our current needs in terms of efficient operations and how this would map to a model where National Grid Initiatives (NGIs) play a significant role.

The issues of 24x7 operations also needs to be discussed with priority.

15:30 coffee break

16:00 Experiment Operations

What is it that the experiments hate most about the current operations setup?

What explicitly is missing from the point of view of the experiments?

What can be done better? What (perhaps?) should not be done at all?

Should we somehow integrate global / experiment operations? e.g. via repeat consoles in the various operations rooms?

Wednesday, 28 November

09:00 → 18:00 Monitoring - What is Required to run Reliable Services?

VRVS details: Saturn

Morning: Outstanding requirements for current projects and discussion of where this might go - e.g. SAM/ gridview, nagios-based prototype, GOCDB, CIC Portal, Experiment Dashboards

Afternoon: discussion of the requirements identified during Tuesday's sessions, building a medium-long term plan.

09:00 Introduction

Speaker: Ian Neilson (CERN)

Slides WLCG_Reliability_Monitoring.pdf WLCG_Reliability_Monitoring.ppt

09:15 Management reporting requirements

Speaker: Alberto Aimar (CERN)

Slides LCG_management_20071128.pdf LCG_management_20071128.ppt

09:35 OSG Requirements

Speaker: Rob Quick (OSG - Indiana University)

Slides OSG_Monitoring_Requirements.pdf OSG_Monitoring_Requirements.ppt

09:55 Requirements from CIC Portal users

Speaker: Gilles Mathieu (IN2P3/CNRS Computing Centre, Lyon, France)

Slides CICportalRequirements_WLCGworkshop112007.pdf CICportalRequirements_WLCGworkshop112007.ppt

10:15 Coffee

10:30 GOCDB Requirements

Slides Gordon-GOCDB-req.pdf Gordon-GOCDB-req.ppt

10:50 EGEE Metrics Implementation Group

Speaker: Etienne Urbah (LAL)

Slides WLCG-Service-Reliability-Workshop-Metrics-Implementation-Group-V8.pdf WLCG-Service-Reliability-Workshop-Metrics-Implementation-Group-V8.ppt

11:10 Site Monitoring Prototype Feedback

Speaker: Emir Imamagic (Unknown)

Slides Site_Monitoring_Prototype_Feedback.pdf Site_Monitoring_Prototype_Feedback.ppt

11:30 Discussion

12:00 lunch break

13:00 Service availability calculation methods - now and the future

Speaker: Piotr Nyczyk (CERN)

Slides SAM-availability-calculation.pdf SAM-availability-calculation.ppt

13:20 Security for Grid Sites

Speaker: Louis PONCET (CERN)

keynote presentaion SecurityWLCG_workshop.key.zip

Slides

• Security_2007_WLCG_WS.pdf
• SecurityWLCG_workshop.pdf
• SecurityWLCG_workshop.ppt

13:40 Use cases for GridMap visualization

Speaker: Max Boehm (Unknown)

Slides GridMaps_UseCases_2007-11-28.pdf GridMaps_UseCases_2007-11-28.ppt

14:00 Models for monitoring

Speaker: James Casey (CERN)

Slides 0711-WLCG-Models_for_Monitoring.pdf 0711-WLCG-Models_for_Monitoring.ppt

14:20 coffee break

14:40 Experiment Critical Services and Monitoring - What's Missing for CCRC'08 (and beyond)?

Speaker: Julia Andreeva (CERN)

16:10 Prioritization of requirements raised during the day

16:40 LHC VO User Support evolution update

Speaker: Maria Dimou (CERN)

Slides http://cern.ch/dimou/lcg/UserSupport/ServiceReliability-UserSupport-20071128.ppt

17:00 Mid-workshop checkpoint

Speaker: Jamie Shiers (CERN)

Slides SoNowWhat.pdf SoNowWhat.pptx

Thursday, 29 November

09:00 → 14:00 Robust Services - Middleware Developers' Techniques & Tips

VRVS details: Plane

Key techniques from middleware / storage-ware developers for making services robust by design

09:00 CASTOR

Speaker: Dennis Waldron (CERN)

Slides WLCG_CASTOR2.pdf WLCG_CASTOR2.ppt

09:30 LFC / DPM

Speaker: Jean-Philippe Baud (CERN)

Slides LFC-DPM_ReliabilityWorkshop.pdf LFC-DPM_ReliabilityWorkshop.ppt

10:00 FTS

Speaker: Paolo Tedesco (Unknown)

Slides FTS_reliability.pdf FTS_reliability.ppt

10:30 coffee break

11:00 Other m/w sessions: BDII, WMS/LB, VOMS, R-GMA, Logging format

12:30 lunch break

14:00 → 18:00 DB application design issues

14:00 DB design (Tips & tricks)

• schema design,
• columns and constraints,
• indexes and partitioning,
• proper use of PL/SQL (bind variables, static and dynamic SQL),
• writing robust applications

Speaker: Dawid Wojcik (CERN)

Slides Workshop_DB_Design.pdf Workshop_DB_Design.ppt

15:30 coffee break

15:50 DB development tools (with demo)

• bentic
• application express
• sql developer </UL

Speaker: Balys Sulmanas (CERN)

Slides DevTools.pdf DevTools.ppt

16:50 DB Monitoring tools (demo)

• session manager
• OEM
• 3D

Speaker: Dawid Wojcik (CERN)

Slides Workshop_Mon_Tools.pdf Workshop_Mon_Tools.ppt

17:30 CORAL

Speaker: Andrea Valassi (CERN)

Slides WLCG_CORAL_2007.pdf WLCG_CORAL_2007.ppt

Friday, 30 November

09:00 → 13:30 DB - performance and tuning issues

VRVS details: Car

09:00 DB performance/tuning in a RAC environment for developers

- Advanced SQL - Optimization - Examples from production

Speaker: Miguel Anjo (CERN)

Slides WLCG_Reliability_db-performance-tuning.pdf WLCG_Reliability_db-performance-tuning.pptx

10:15 coffee break

10:40 DB performance tuning in a RAC environment for administrators

• Service architecture for performance
• Monitoring and reporting performance issues
• Reactive performance tuning, case studies
• New HW sizing and capacity planning

Speaker: Luca Canali (CERN)

Slides WLCG_Oracle_perf_for_admin_Luca_Nov07.pdf WLCG_Oracle_perf_for_admin_Luca_Nov07.ppt

12:00 Dataguard for RAC Migration

Speaker: Jacek Wojcieszuk (CERN)

Paper CERN_Dataguard_Migration2.pdf

Slides DataGuard_for_RAC_migrations_v2.pdf DataGuard_for_RAC_migrations_v2.ppt

12:30 lunch break

13:30 → 17:30 DB - service issues

13:30 Service plans for 2008

Speaker: Maria Girone (CERN)

Slides DBservices_2008_SRW.pdf DBservices_2008_SRW.ppt

13:45 Service Recommendations

• Security of machines and authentication techniques
• How to manage your logs (listener.log, crs logs, alert logs etc)
• How to manage your Oracle environment (host environment)
• A quick recap on the backup emails (from the talk at CNAF)
• Managing your targets in Grid Control Security of machines and authentication techniques

Speaker: Gordon Brown (CCLRC)

Slides CERN_WLCG_2007_1.pdf CERN_WLCG_2007_1.ppt

14:45 streams service review

Speaker: Dafonte Perez Eva (CERN)

Slides StreamsServiceReview.ppt

15:45 coffee break

16:05 adding/removing nodes plus demo

Speaker: Jacek Wojcieszuk (CERN)

Slides RAC-adding_and_removing_nodes.pdf RAC-adding_and_removing_nodes.ppt