WLCG-OSG-EGEE Operations meeting

Europe/Zurich
28-R-15 (CERN conferencing service (joining details below))

28-R-15

CERN conferencing service (joining details below)

John Shade (CERN)
Description
grid-operations-meeting@cern.ch
Weekly OSG, EGEE, WLCG infrastructure coordination meeting.
We discuss the weekly running of the production grid infrastructure based on weekly reports from the attendees. The reported issues are discussed, assigned to the relevant teams, followed up and escalated when needed. The meeting is also the forum for the sites to get a summary of the weekly WLCG activities and plans
Attendees:
  • OSG operations team
  • EGEE operations team
  • EGEE ROC managers
  • WLCG coordination representatives
  • WLCG Tier-1 representatives
  • other site representatives (optional)
  • GGUS representatives
  • VO representatives
  • To dial in to the conference:
    a. Dial +41227676000
    b. Enter access code 0148141

    OR click HERE
    (Please specify your name & affiliation in the web-interface)

    Click here for minutes of all meetings

    Click here for the List of Actions

    • Monday, 20 July
      • 16:00 16:20
        EGEE Items 20m
        • <big>Central Grid-Operator-on-Duty (c-COD) handover</big>
          Form ROC Italy to ROC CentralEurope
        • last week we handled some ticket expired and old alarms as usual.
        • there is just a site (UKI-LT2-UCL-CENTRAL) that is in downtime since June and has a ticket opened since May 22nd.
        • I've suggested the ROC_UKI to change the site status in "uncertified" in order to stop the raising of the alarms, until the site will be ready to come back in production. At least fo now, ROC_UKI has just changed the expiration date of the ticket to Aug 18th (the end of the downtime of gw-4.ccc.ucl.ac.uk). The downtime of the other host will end on July 30th.
  • <big> PPS Report & Issues </big>
    Please find Issues from EGEE ROCs and general info in:
    https://twiki.cern.ch/twiki/bin/view/LCG/OpsMeetingPps
    1. gLite3.2.0-PPS-UPDATE04 is available for pre-production from this morning. It contains new releases for BDII, LFC mysql and DPM mysql in SL5. It also contains new versions of yaim core and yaim cliernts.
    2. <big> gLite Release News</big>
      Please find gLite release news in:
      https://twiki.cern.ch/twiki/bin/view/LCG/OpsMeetingGliteReleases

      1. Release of UPDATE 50 to gLite 3.1: gLite-WN. New version of grid-cm-* packages to remove /opt/glite/lib/python/logging. This is to address GGUS ticket 50148, only happening in glite-WN version 3.1.31-0. In fact this private logging version was only ever required on SL3 and can cause problems for people using private python 2.5 versions with the supplied 2.3 versions.
      2. Release of UPDATE 51 to gLite 3.1, including new release of yaim clients, new version of DPM, new version of LFC, new version of yaim LFC
      3. <big> EGEE issues coming from ROC reports </big>
        1. ROC UKI: Published to LCG-ROLLOUT but worth mentioning here for those who missed it:
          Eventually RAL Tier-1 found that the cause of the wmproxy issue they had (not finding the VOMS attributes) was that the mod_gridsite part of the WMS wmproxy service does not recognise new format VOMS credentials . The new style VOMS credentials are created when using the --newformat switch on the VOMS server voms_install_db command see https://edms.cern.ch/file/973684/1/voms-guide.pdf (page 19). Details of the differences between old and new formats can be found here:
          https://savannah.cern.ch/bugs/?10894
          https://savannah.cern.ch/bugs/?17273
          The new /correct VOMS credentials are not that new , the above bug reports are from 2005, but it looks like many (presumably all other?) VOMS servers are still issuing the incorrect/old format.
          There is now a bug report about the wmproxy/mod_gridsite problem we found here https://savannah.cern.ch/bugs/?53314 . We can t move forward much with our plans for the WMS until we ve found a fix for this problem, but at least we know the cause.

      4. <big>Grid Service Interventions </big>
        Link to CIC Portal (broadcasts/news), scheduled downtimes (GOCDB) and CERN IT Status Board
        Please consult the URLs above for details.

      5. <big> gLite 3.0 VOMS servers - VOMS-client incompatibility on the way! </big>

        The following VOMS servers are running gLite 3.0 versions:
        • https://grid12.lal.in2p3.fr:8443/vomses ( 16 VOs )
        • https://voms.gridpp.ac.uk:8443/vomses ( 20 VOs )
        • https://cagraidsvr10.cs.tcd.ie:8443/vomses ( 11 VOs )
        • https://grids13.eng.it:8443/vomses ( 4 VOs )
        • https://voms.kek.jp:8443/vomses ( 8 VOs )
        • https://voms.grid.sinica.edu.tw:8443/vomses ( 6 VOs )
        • https://glite-io.scai.fraunhofer.de:8443/vomses ( 1 VO )
        • https://voms.ndgf.org:8443/vomses ( 9 VOs )
        • https://skurut19.cesnet.cz:8443/vomses ( 6 VOs )

        NB: This list may not be complete.

        As announced on 3 July*, there is a version of the VOMS-client currently in certification which is incompatible with the 1.7.x (gLite 3.0) versions of the VOMS server.

        These VOMS servers must be upgraded as soon as possible.


        * https://cic.gridops.org/index.php?section=roc&page=broadcastretrievalC&step=2&typeb=C&idbroadcast=41703
      6. 16:20 16:30
        OSG Items 10m
        Speakers: Maria Dimou, Rob Quick
      7. 16:30 16:35
        Review of Action Items 5m
      8. 16:35 16:40
        AOB 5m