Regional Operations Meeting DECH

Friday 9 Feb 2007, 15:00 → 16:00 Europe/Zurich

FZK

Sven Hermann

Telephone conference. Details for connecting to the conference were distributed through the regional mailing list.

Minutes for Regional Operations Meeting DECH (February 9th, 2007) Attendance: Sven Hermann <chair>, Clemens Koerdt, Günter Grein (FZK) Hans-Gunter Borrmann (Uni Freiburg) Christian Peter (ITWM) Andreas Haupt, Kai Leffhalm (DESY-ZN) Alessandro Usai (CSCS) Uwe Ensslin, Christoph Wissing, Andreas Gellrich (DESY-HH) Kläre Cassirer, Horst Schwichtenberg (SCAI) Apologies: (Uni Wuppertal) (GSI) Missing: (MPPMU) (Uni Karlsruhe) (Uni Dortmund) (RWTH-Aachen) 1. Introduction Last meetings minutes: no comments Result of Partner review: Overall conclusion was very positive, especially that all partners contributed to the achievements in our region and how much we achieved Only criticism/suggestion: distribute more of the tasks to the partners and make known positive developments and successes (in weekly reports or other) General conclusion: very fruitful and successful meeting (thanks to all of you) Announcements: New PPS portal announced (link on the agenda page) Update 13 released with SL4 WN package, but no 64bit support yet. Still quite an issue porting to 64bit Until now only tarballs but no RPMs yet. No realistic timescale yet for porting Hints on how to correctly install info scheduler by Jeff T. (see link on agenda page) Action Items: Action items related to the partner review can now be closed. (Thanks to everybody who contributed to make this a success.) PPS certification-> GSI has added resources and is monitoring their nodes. ITWM has registered resources in GocDb. (start monitoring: #! ITWM, get infosystem and SAM tests working: #! GSI, ITWM) Dech VO: Supported by all sites in region? -> VOMS server is working again. Started to test sites again: CSCS , Freiburg now supporting correctly, ITWM still a problem. (#! ITWM). Some sites still need to be tested and will be informed soon on their status. Regional SFT-Server: -> due to partner review and series of meetings no further progress, will receive high priority in the coming weeks (!# CK). 2. Round the Sites CSCS ------ participation in COD meeting was very useful Alessandro will do the coming COD shifts still working on the new cluster in production hopefully by end of February DESY ----- not much happened apart form usual production running experienced many WMS troubles (see DECH ticket 1581 for details). Always surprised that nobody (and especially developers) have noticed those troubles before. Updates work quite fine, positive development when you think of the past big bang releases Discussion on how best to handle the SL4 topic how do others think about it? any hacks and any changes not trivial for big sites (tarballs not that much welcomed) AU: WNs package available, worked with YAIM (but not yet in production) CW: at DESY we were experiencing troubles with it AU: apt repository of PPS was used. AU: so confident that package will be there very soon. Not suggesting using PPS repository for production SH: will again address the SL4 issue in next weeks Operations Meeting (!# SH, done at time of writing) AG: Are all VOs able to work under SL4? Might not be trivial! How do others address this? SH: At FZK we plan to deploy only a few WNs under SL4 and then get feedback from the VOs AU: PPS service can be used by the VOs to test their software under SL4. SH: This is exactly the way it should be. AU: Are we represented in this PPS SL4 activity? SH. Unfortunately, not yet in our region. CW: We had installed SL4 on some WNs, but not yet officially announced to PPS (..and not sure if we should do that) DESY: WMS troubles DECH#1581 to be brought forward to next ops meeting. (!#SH, done at the time of writing) DESY-ZN ----- Experienced Atlas software directory problems Atlas reinstalled software this week and are now back in production RGMA not working for a couple of weeks Suppose there is a central problem SH: raised APEL test problem last ARM meeting, but got no update on this. FZK ----- rm-test failure rate reduced dramatically. Turned out to be a configuration issue. Ops test trying to write onto tape instead of disk. Stalling (LHCb) transfers now aborted using timeouts. Soft- and Hardware maintenance next Monday (downtime) ITWM ----- production system is running fine. accounting data problems also, no time yet to investigate Central problem or site problem PPS: some more days needed before starting the monitoring. Will provide replica web front end for GOCDB, will go into production with the new release (still a test version) SCAI ----- production system running fine PPS also, problem Dortmund ----- last weekend headnode of small dCache installation crashed Karlsruhe (feedback received before the actual meeting) ----- got some maintenance work done Freiburg ----- had a longer downtime due to maintenance replaced gLite with lcg CE dCache installation also all seems to work fine now atlas seem to work fine again, but still smaller problems with dCache (access to stored files). 3. Feedback to the TCG There was some discussion with a couple of suggestions proposed by HS: Waiting for your input! Use this chance to bring your ideas forward directly to the TCG! For example: Do you need architecture description? or new functionalities? web services? ROC-support?, specific MPI versions?, any issues with compilers? any issues with underlying system?, ... Do we have to change something, do we need anything? Freiburg: Main concern is need of SL4 and 64bit (cant use the machines) Whats actually done by the users on the cluster? What do the users want? Please ask your users and come back with their suggestions (!# all) 4. COD Successful duty shift two weeks ago shared ticket with the backup team Work sharing with backup team successful and eased workload considerably provided feedback at COD-12 in Karlsruhe this week strong presence of COD DECH in the meeting creating the contact to the developer teams Christian Peter attacking GocDb failover integrated new team member Alessandro Usai who is replacing Peter Kunszt. 5. ROC-On-Duty Handover DESY to FZK DESY: nothing much to report situation has improved, especially the number of very old tickets. However, asking people directly usually does not result in a response SH: present statistics here in this meeting. Is this possible? GG: I do not think its a problem. Will prepare a template for next handover. (!# GG) GG: There are two very old tickets (from November). Has DESY any ideas what is wrong with them? DESY: There are no news on the ticket concerning Dortmund supporting the DECH VO. DESY: The other ticket is with MPPMU. They usually do not respond. CK: In case sites are not responding to question from ROC-on-duty forward to either me or Sven. MPPMU ticket will be tackled by me (!# CK) DESY: Tickets that go to developers. Usually do not give any results. (e.g. DECH#986) CK: Report those tickets in your weekly reports so that we are aware of them and can escalated them to the operations meetings if necessary. 6. AOB ------ HS: We expected to receive some TPM training. How is this organized? GG: On GGUS homepage you can find the dates of the scheduled trainings. HS: Did GSI already receive TPM training? SH: Need to clarify this issue with Torsten, who is coordinating this effort (#! TA)

15:00 → 15:05 Introduction

Announcements - Comments on last meetings minutes - Result PAR DECH Review (EGEE internal: see documents,meeting) - New PPS Portal - Production update 13 released. SL4 Workernode released! update - hints for correct installation of lcg-info-dynamic-scheduler (by Jeff Templon, see minutes) Action Items

Speaker: Sven Hermann

15:05 → 15:40 Round the sites

Speaker: ALL

15:40 → 15:45 Feedback to the TCG

Speaker: Horst Schwichtenberg

15:45 → 15:50 Status COD DECH

- Organisation - Report about shift (if applicable)

Speaker: Clemens Koerdt

15:50 → 15:55 Status ROC-DECH-On-Duty-Support

- Handover

Speaker: DESY -> FZK

15:55 → 16:00 AOB