EGEE-SA1-SWE Meeting

→ Europe/Zurich
VRVS Virtual Room (http://www.vrvs.org)

VRVS Virtual Room (http://www.vrvs.org)

Pacheco, Andreu
Description
Present: A. Pacheco (PIC), C. Borrego (PIC), J. Lopez (CESGA), P. Rey (CESGA), A. Casajus (UB), G. Merino (PIC), R. Graciani (UB), F. Bernabve (CESGA), G. Losilla (BIFI), A. Fernandez (IFIC), J. Sanchez (IFIC), M. Rodriguez (PIC), M. Sanchez (USC), K. Neuffer (CAB-INTA), E. Huedo (CAB-INTA) Apologies: M. David (LIP)
    • 12:00 → 12:15
      Briefing of activities 15m VRVS

      VRVS

      Speaker: Pacheco, Andreu (IFAE/PIC)
      more information
    • 12:05 → 12:15
      Operational status from sites 10m VRVS

      VRVS

      * From February 1st 2005 all sites must report by mail their operational status each friday if they want Gonzalo Merino to report with accuracy their status to EGEE Operations Meeting * Since the people from CERN is asking the ROCs to write the monday's 11 report following a sort of template, I think it will be useful that the SWE Site Reports that site managers are sending on friday to this list, try to follow a similar template as well. I suggest then that these reports are produced following the template available at: http://services2.pic.org.es/devel/download/txt/SR-template.txt * Reports should be sent to: egee.swe.roc.contact@pic.es The reports from SWE are appended to the agenda page of the weekly operation meetings: http://agenda.cern.ch/displayLevel.php?fid=258
      Speaker: Merino, Gonzalo (PIC)
      more information
      • CESGA-EGEE 15m
        * REported by Javi Fontan * Power problem at CESGA during the week, they reported site schedulled shutdown but finally no power was really out. * Migrated to LCG 2.4
        Speaker: Fontan, Javier
      • CIEMAT-LCG2 15m
        * Reported by mail. Weekly Operations Status Report: CIEMAT-LCG2 Reporting Period: 18/04/05 to 22/04/05 Availability of Core Services Hosted in the Site ================================================== There is no core services running at CIEMAT at this moment Non-Functional Time ==================== None Scheduled Down Time =================== * Scheduled Maintenance During the Reporting Period Downtime from 18/04/2005 10:00 to 21/04/2005 18:00 for upgrade to LCG-2.4.0 * Scheduled Maintenance During the Next Reporting Period None Major Operational Issues Encountered During the Reporting Period ================================================================ We have completed the upgrade of our cluster to 2.4.0. We have fixed several problems encountered after running site functional tests (port 2170 closed, replica-manager failed). Apel was upgraded to 3.4.43 and we have corrected our configuration files. Upgrade to Scientific Linux 3 (or equivalent) ============================================= By when will all service nodes in the Site be upgraded? Completed, now Scientific Linux 3.0.4 By when will the WNs in the Site be upgraded? Completed, now Scientific Linux 3.0.4 Points to Raise at the Operations Meeting ========================================= None
        Speaker: Calonge, Javier
      • CNB-LCG2 15m
        * No report. * CNB is not reporting accounting. They fail the RGMA test allways.
        Speaker: Merino, Angel
      • IFCA-LCG2 15m
        * No report *
        Speaker: Cano, Dani
      • IFIC-LCG2 15m
        * Javi Sanchez reporting. * Migration to LCG 2.4 last wednesday, with minor problems.
        Speaker: Sanchez, Javier
      • INTA-CAB 15m
        * Reported by Kai Neuffer * RGMA problem. Open ticket. * Pending upgrade of Apel. Weekly Operations Status Report: INTA-CAB Reporting Period: 18/04/05 to 25/04/05 Availability of Core Services Hosted in the Site ================================================ No core services hosted in this site. Non-Functional Time =================== 0 days. Scheduled Down Time =================== * Scheduled Maintenance During the Reporting Period 0 days. * Scheduled Maintenance During the Next Reporting Period 0 days. Major Operational Issues Encountered During the Reporting Period ================================================================ R-GMA and Apel problems after upgrade. Upgrade to Scientific Linux 3 (or equivalent) ============================================= Already done. Points to Raise at the Operations Meeting ========================================= None
        Speaker: Huedo, Eduardo
      • LIP-LCG2 15m
        * Reported by mail as today is holiday at Portugal. * New installed LFC. Mario requests to all SWE site managers to test the SWETEST LFC. * Javi Fontan from CESGA has tested successfully the new LFC. Weekly Operations Status Report: LIP Reporting Period: 16/04/05 to 22/04/05 Availability of Core Services Hosted in the Site ================================================== RB: rb02.lip.pt LCG-BDII: ii02.lip.pt RLS: se01.lip.pt LFC: se01.lip.pt Non-Functional Time ==================== [CT] [JS] Failed 2005-04-17/20/21/22 maui/torque completelly unstabe at a certain point with more than 800 queued jobs. Jobs were drained and the system restarted [OK] 3 days Scheduled Down Time =================== Major Operational Issues Encountered During the Reporting Period ================================================================ SFT failure due to the queues being full (see reason above) problem solved. Reconfiguration of maui/torque planned next week. Upgrade to Scientific Linux 3 (or equivalent) ============================================= All nodes in SLC304 Points to Raise at the Operations Meeting ========================================= LFC has ben deployed for "swetest", "atlas" and "cms" VO's Its up and running OK, at least until someone complains!! All SWE sites should move fast to use the LFC, the swetest catalog was migrated from RLS to LFC. The interaction/usage between the WMS and the LFC has to be checked. Network measurement is being done Sites already performing measurements LIP CESGA Improvement of the web pages ongoing (have taken to second priority).
        Speaker: David, Mario
      • IFAE 15m
        * Reported by Carlos Borrego. * Problem with Castor. Site down until this evening.
        Speaker: Borrego, Carlos
      • PIC 15m
        * Reported by Carlos Borrego. * Schedulled downtime. Upgrading LCG.
        Speaker: Borrego, Carlos
      • UAM-LCG2 15m
        Weekly Operations Status Report: [ UAM-LCG2 ] Reporting Period: [ 15/04/05 ] to [ 22/04/05 ] Availability of Core Services Hosted in the Site ================================================== MyProxy [OK] VOServer [OK] RB [OK] BDII [OK] Non-Functional Time ==================== Scheduled Down Time =================== * Start of downtime --> 2005-04-17 13:00:00 * End of downtime --> 2005-04-19 14:00:00 * Description add new default SE (grid002.ft.uam.es) and extra SE (grid004.ft.uam.es). grid002 is a DCACHE SE with 3 TB grid004 is a classic SE with 1,5 TB Major Operational Issues Encountered During the Reporting Period ================================================================ none Upgrade to Scientific Linux 3 (or equivalent) ============================================= I am working in a test cluster with SL304 and LCG-2_4_0 By when will all service nodes in the Site be upgraded? [ May ] By when will the WNs in the Site be upgraded? [ May ] Points to Raise at the Operations Meeting ========================================= none <comments> - RB and BDII are two diferent machines - UAM-LCG2 has 2 SEs: DCACHE SE (default): 3 TB Classic SE (extra): 1,5 TB
        Speaker: Pardo Navarro, Juan Jose
      • UB-LCG2 15m
        * Ricardo Graciani reporting. * Start of upgrade LCG + SCL Weekly Operations Status Report: UB Reporting Period: 17/04/2005 to 23/04/2005 Availability of Core Services Hosted in the Site ================================================ Non Core Service in the Site Non-Functional Time ==================== Scheduled Down Time =================== None Major Operational Issues Encountered During the Reporting Period ================================================================ None Upgrade to Scientific Linux 3 (or equivalent) ============================================= By when will all service nodes in the Site be upgraded? By when will the WNs in the Site be upgraded? (together with update to 2.4.0) Start during this week. Points to Raise at the Operations Meeting ========================================= None
        Speaker: Graciani, Ricardo
      • UPV-GRyCAP 15m
        Weekly Operations Status Report: [ UPV-GRyCAP ] Reporting Period: [ 18/04/05 ] to [ 22/04/05 ] Availability of Core Services Hosted in the Site ================================================== RB, BDII: apis.dsic.upv.es [OK] Non-Functional Time ==================== - Total days unavailable: 2 days 21-04-2005 & 22-04-2005 -> Problems with the MDS service. Scheduled Down Time =================== * Scheduled Maintenance During the Reporting Period - Total days unavailable 0 * Scheduled Maintenance During the Next Reporting Period - Estimated number of days for which the site will be unavailable 0 days Major Operational Issues Encountered During the Reporting Period ================================================================ Problems with the MDS service. The information published by the GRIS of the CE in the GIIS service, disappears intermittently. Upgrade to Scientific Linux 3 (or equivalent) ============================================= By when will all service nodes in the Site be upgraded? Already done (SL 3.0.4 & FC2) By when will the WNs in the Site be upgraded? Already done (FC2) Points to Raise at the Operations Meeting ========================================= none
        Speaker: Caballer, Miguel
      • USC-LCG2 15m
        * Manuel Sanchez reporting. * Start of upgrade. Power glich last week. * Site configuration problems relative to the grid mapfile. Now corrected. Reporting Period: 18/04/2005 to 25/04/2005 Availability of Core Services Hosted in the Site ================================================ None Deployed Non-Functional Time =================== None Scheduled Down Time =================== From Wed 20 Apr 2005 20:00 to Fri 22 Apr 2005 09:00 Major Operational Issues Encountered During the Reporting Period ================================================================ Upgraded to LCG 2.4.0 SFT tests are failing with a job submission error. The problem is under investigation Upgrade to Scientific Linux 3 (or equivalent) ============================================= By when will all service nodes in the Site be upgraded? Already there By when will the WNs in the Site be upgraded? Already there Points to Raise at the Operations Meeting ========================================= None
        Speaker: Sanchez, Manuel
      • GGUS EGEE SA1 SWE open tickets for sites 15m
        Speaker: David, Mario (LIP)
      • BIFI-LCG 15m
    • 12:10 → 12:15
      APEL Status Report 5m VRVS

      VRVS

      * Pablo Rey reporting. * IFCA y CNB no publican accounting. * Todos los que han actualizado LCG a 2.4 no publican excepto CESGA por un bug. Upgrade manually to Apel 2.4.43 and modify the cron procedure adding a environment variable at the top of the script. APEL STATUS REPORT == Period == From April 16 to April 22 == Summary == April 21 Flexible Archiver has been down. This avoids publishing accounting data during these days even if Apel is working fine in some sites. Like the last week, during this week, several sites have been in maintenance to update to LCG2.4.0. By other hand, taking into account the received monitoring alerts, other sites have had problems with their machines. We want to emphasize that all sites that have updated to LCG2.4.0 are having problems with their accountind data: they are not publishing it. These sites have to update Apel to version 3.4.43 after the installation of LCG2.4.0 (we have sent an e-mail during this week about this issue). They also have to check the configuration file of Apel (/opt/edg/etc/edg-rgma/apel.xml) because this file will be rewrite by the installation procedure. == Status of each SWE site == Site: CESGA Solved Problems of Last Report: Yes Missing Accounting Entries: None Publishing User Name Information: Yes Actions: None. They have installed the new version of Apel so they should publish the old data the next time that Apel work. Other Comments: They have updated to LCG2.4.0 and also they have updated to Apel 3.4.43. Site: CIEMAT Solved Problems of Last Report: Yes Missing Accounting Entries: Publishing User Name Information: NO. April 15 have not published the user name. Actions: Check the configuration of Apel: They are reprocessing their data and they are not publishing user name. Other Comments: Update to Apel 3.4.43 and check the configuration of Apel before run it. Site: ifae Solved Problems of Last Report: NO Missing Accounting Entries: Since 09/04/05 Publishing User Name Information: Yes Actions: Solved Problems of Last Report. JoinProcessor Reprocess Other Comments: None Site: IFCA Solved Problems of Last Report: NO Missing Accounting Entries: Since 15/01/05 Publishing User Name Information: Yes Actions: Solved Problems of Last Report. Reprocess Other Comments: None Site: IFIC Solved Problems of Last Report: NO Missing Accounting Entries: 21, 22 Publishing User Name Information: Yes Actions: Solved Problems of Last Report. They have installed the new version of Apel so they should publish the old data the next time that Apel work. Other Comments: They have updated to LCG2.4.0 so they should update to Apel 3.4.43 and check the configuration of Apel before run it. Site: INTA Solved Problems of Last Report: NO. Missing Accounting Entries: Since 09/04/05 Publishing User Name Information: Yes Actions: None. They have installed the new version of Apel so they should publish the old data the next time that Apel work. Other Comments: They have updated to LCG2.4.0 so they should update to Apel 3.4.43 and check the configuration of Apel before run it. Site: LIP Solved Problems of Last Report: NO Missing Accounting Entries: Since 09/04/05 Publishing User Name Information: Yes Actions: None. They have installed the new version of Apel so they should publish the old data the next time that Apel work. Other Comments: They have updated to LCG2.4.0 and also they have update to Apel 3.4.43. They should check the configuration of Apel before run it. Site: pic Solved Problems of Last Report: NO Missing Accounting Entries: Since 13/04/05 Publishing User Name Information: Yes Actions: Solved Problems of Last Report. JoinProcessor Reprocess. Other Comments: They have updated to LCG2.4.0 so they should update to Apel 3.4.43 and check the configuration of Apel before run it. Site: PIC-LCG2 Solved Problems of Last Report: Since 02/03/05 they are publishing user information, but they have not publish the lost user information. They don't reprocess. Missing Accounting Entries: Since 13/03/05 Publishing User Name Information: Yes Actions: Solved Problems of Last Report. JoinProcessor Reprocess Other Comments: At this moment there are three sites names for PIC: this, pic and ifae. This site name was changed by ifae in the monitoring tools, so will you publish more data with this name?. Could we stop the cheching of this site name? Site: UAM Solved Problems of Last Report: NO Missing Accounting Entries: 21 Publishing User Name Information: Yes Actions: Solved Problems of Last Report. JoinProcessor Reprocess Other Comments: None Site: UB Solved Problems of Last Report: NO Missing Accounting Entries: 21 Publishing User Name Information: Yes Actions: Solved Problems of Last Report. JoinProcessor Reprocess Other Comments: None Site: UPV Solved Problems of Last Report: Yes Missing Accounting Entries: None Publishing User Name Information: Yes Actions: None. Other Comments: None Site: USC Solved Problems of Last Report: Yes Missing Accounting Entries: 21, 22 Publishing User Name Information: Yes Actions: None. They have installed the new version of Apel so they should publish the old data the next time that Apel work. Other Comments: They have updated to LCG2.4.0 so they should update to Apel 3.4.43 and check the configuration of Apel before run it. == Accounting Support == Accounting Web Page: http://www.egee.cesga.es/EGEE-SA1-SWE/accounting/ Accounting FAQ: http://www.egee.cesga.es/EGEE-SA1-SWE/accounting/faq.html Accounting Statistics: http://www.egee.cesga.es/EGEE-SA1-SWE/accounting/reports/ CPU Normalization: http://www.egee.cesga.es/EGEE-SA1-SWE/accounting/normalization.html Accounting Guides: http://www.egee.cesga.es/EGEE-SA1-SWE/accounting/guides/ Contact e-mail: egee-admin@cesga.es
      Speaker: Rey Mayo, Pablo (CESGA)
    • 12:15 → 12:30
      Action lists 15m VRVS

      VRVS

      Speaker: Pacheco, Andreu (IFAE)
      • 5.1.17.4 Andreu Pacheco to update the SWE Execution Plan (PENDING) 15m
        Speaker: Pacheco, Andres (PIC)
      • 5.1.17.7 CIEMAT, CNB, IFIC, IFAE, UAM and USC to upgrade worker nodes to Scientific Linux 3.0.2 (PENDING) 15m
        Speaker: Merino, Gonzalo (PIC)
      • 5.1.17.8 CNB to upgrade Apel version 3.4.39-1 (PENDING) 15m
        Speaker: Lopez Cacheiro, Javier (CESGA)
      • 5.3.7.1 Miguel Cardenas to check if forensic analysis after intrusion is mandatory and who can do it (NEW) 15m
        Speaker: Cardenas, Miguel (CIEMAT)
      • 5.3.7.2 Farida Fassi and Mohammed Kaci to setup the initial user support web site (OPEN) 15m
        Speaker: Fassi, Farida (IFIC)
      • 5.3.7.4 Mario to deploy gmdat monitoring tools in SWE (OPEN) 15m
        Speaker: David, Mario (LIP)
    • 12:20 → 12:35
      GGUS Open Tickets for SWE Federation 15m VRVS

      VRVS

      Speaker: Pacheco, Andreu (PIC)
      more information
      • 2273 - GriCAP-UPV - Bad configuration of the /opt/edg/var/info/ area in the CE 15m
        25.Apr. 2005, 09:07(UTC) by wsuser Making edg-gridftp-ls -v gsiftp://ramses.dsic.upv.es/opt/edg/var/info/dteam It shows: rw-r--r-- 1 root 0 Mar 14 11:33 dteam.list Exactly the same for the rest of VOs. This is bad configured. The 1st time the tool lcg-ManageVOTag is used creates this file with write permisions for the sgm persons. So after the configuration of the SE, this *.list files should not exist. In this sense experiments that want to install software in this site will not be able to publish the corresponding tag in the Information System and the following production in this site will not arrive.
      • 2267 - IFAE - replication failed Problems with SE 15m
        25.Apr. 2005, 08:43(UTC) by cicuser ----------Affected site: ifae---------- *** VERY URGENT ***
      • 2258 - PIC - Got a job held event, reason: Unspecified gridmanager erro 15m
        22.Apr. 2005, 13:59(UTC) by wsuser Hi, Affected CE : ce01.pic.es:2119/jobmanager-lcgpbs-biomed My jobs cannot be execute on this CE, I have the following error message before the resubmission of the job : Got a job held event, reason: Unspecified gridmanager error Perhaps, a problem with the CA RPMs which are not up to date, but it is not the usual message for this problem. *** URGENT ***
      • 2253 - CIEMAT - replication failed 15m
        22.Apr. 2005, 11:02(UTC) by cicuser ----------Affected site: CIEMAT-LCG2---------- 22.Apr. 2005, 22:15(UTC) by AR_ESCALATOR ----------Affected site: CIEMAT-LCG2---------- 23.Apr. 2005, 22:15(UTC) by AR_ESCALATOR ----------Affected site: CIEMAT-LCG2---------- 24.Apr. 2005, 22:15(UTC) by AR_ESCALATOR ----------Affected site: CIEMAT-LCG2---------- *** URGENT ***