Go to the Indico Home Page Oldest event Older event Up to category Newer event Newest event Printable version
  • Compact style
  • Indico style
  • Indico style - inline minutes
  • Indico style - numbered
  • Indico style - numbered + minutes
  • Indico Weeks View
    • Switch to the management area of this event
    Login

    WLCG-OSG-EGEE Operations meeting

    → Europe/Zurich
    28-R-15 (CERN conferencing service (joining details below))

    28-R-15

    CERN conferencing service (joining details below)
    Description
    grid-operations-meeting@cern.ch
    Weekly OSG, EGEE, WLCG infrastructure coordination meeting.
    We discuss the weekly running of the production grid infrastructure based on weekly reports from the attendees. The reported issues are discussed, assigned to the relevant teams, followed up and escalated when needed. The meeting is also the forum for the sites to get a summary of the weekly WLCG activities and plans
    Attendees:
  • OSG operations team
  • EGEE operations team
  • EGEE ROC managers
  • WLCG coordination representatives
  • WLCG Tier-1 representatives
  • other site representatives (optional)
  • GGUS representatives
  • VO representatives

    • To dial in to the conference:
    a. Dial +41227676000
    b. Enter access code 0157610

    OR click HERE

    list of actions
    Minutes
      • 16:00 → 16:05
        Feedback on last meeting's minutes 5m
        more information
      • 16:05 → 16:55
        EGEE Items 50m
        • <big> Grid-Operator-on-Duty handover </big> 5m
          From ROC UK/I (backup: ROC Italy) to ROC AsiaPacific (backup: ROC Russia)

          Lead team handover
          Tickets:
          Created: 29
          Open tickets :
          1st email: 16
          2nd email: 10
          Quarantine: 7
          Site OK: 41
          Unsolvable: 2
          For Ops Meeting: 1
          Close:

          Notes:

        • The only site to be raised for discussion at the Operations Meeting is PreGR-01-UoM (SEE ROC), tkt: #15574 (The other problem for same site is now in quarantine. )

  • <big> PPS reports </big> 5m
    PPS reports were not received from these ROCs: AsiaPacific; France; Italy; North Europe; Russia; SE Europe
  • Reminder to ROCs to follow the agreed procedure for new sites (URL).
    We are seeing several new sites in "candidate" and "uncertified" which we have never heard of:
    1. Spain, UCM, Candidate
    2. Cyprus, CY-02, Candidate
    3. Israel, IL-BGU-PPS, Uncertified
    4. Russia, SU-Protvino-IHEP, Uncertified
  • New PPS web portal: http://egee-pre-production-service.web.cern.ch/egee- pre-production-service/
    •
    Speaker: Nicholas Thackray (CERN)
  • <big> EGEE issues coming from ROC reports </big> 10m
    Reports were not received from these ROCs: SE Europe; US-ATLAS; US-OSG
    1. [Information] As SRM service is frequently crashing because of overload, it will be moved on a more powerful hardware on next wednesday. So, the SRM service (ccsrm.in2p3.fr) won't be unavailable from 14/02-17:00 to 15/02-10:00. (France ROC)


    2. SL4 migration schedule urgently needed. Sites want to know: When will SL4 WN be fully available, i.e. RPMs? When 64-bit Support? This should be made transparent e.g. in the Wiki (https://twiki.cern.ch/twiki/bin/view/EGEE/EGEEgLiteWorkPlans). Instead, SL4 and 64-bit appear under "activities ... supposed to be addressed by all developers" though a time schedule is of course needed but not given! Also, under https://twiki.cern.ch/twiki/bin/view/EGEE/EGEEgLite64bits shows status seven months old (!) (DECH ROC)

      3. Answer from SA3: There are priorities, but there is no timescale. The tarball is now available for SL4 WN and UI. This is SL3 binaries plus appropriate dependencies.
      For the native SL4 WN and UI, we need the build. We don't have it yet, but it is the number one priority now for SA3 and JRA1 (for 32bit). The live status is here:
      http://etics.cern.ch:8080/WebReactor/?project=glite-node- WN_branch_3_1_0_x86_slc_4
      As you can see, we are pretty close. After we have the build, there will be at least a month of testing and configuration work. During that time, x86_64 support will be worked on for the client nodes, along with 32 bit builds of the services.
      For the status of the two links you mention, please contact Claudio Grandi.

    3. Trouble with gLite WMS in production -> 100k files in /tmp ! clean-up issue. https://gus.fzk.de/ws/overview.php?ticket=18270 (DECH ROC)


    4. APEL accounting: some sites complain about data for 1 month missing. SAM- APEL test should monitor this, but isn''''t working. Is it possible to get an update of the status of the SFTest/of APEL? (DECH ROC)

      5. Answer from Dave Kant: How do those sites determine that data is missing for 1 month? Did they query the archiver using RGMA, or did they go to the CESGA portal? Data for jobs which completed more than two weeks ago are periodically stripped off the archiver every night and stored on a standalone database. History queries via RGMA would not show data older than about two-weeks. The CESGA portal reports summaries of whats in the standalone database (currently 24 million job records).
      When the existing SFT tests alerted these sites that they had not published for 3 days, did these sites examine their apel.logs? Did they submit a GGUS ticket asking for help?
      Any site that hasn't published data for over one month must be addressed. CESGA and RAL are planning a new set of APEL tests for SAM that will address site synchronisation issues. These new tests are not based on RGMA queries to the archiver, but on MySQL queries to the stand-alone system. The test data would then be published into SAM via the web service URL and available through RGMA queries, and the CESGA portal. The RGMA queries would be much faster as the table would contain only a few rows per site.

    5. Dissimilarites between gridview/sam and the results on CIC report page. (PPS and Prod mixed?) GridKA has reported this repeatedly since a few weeks already. (DECH ROC)


    6. The SAM tests for the glite-CEs quite often fail the JobSubmission with the error:
      reason = Got a job held event, reason: "The job attribute PeriodicHold expression ''''Matched =!= TRUE && CurrentTime > QDate + 900'''' evaluated to TRUE"
      Is this a known problem? (SW Europe ROC)


  • <big> Grid Storage System Deployment (GSSD) </big> 10m
    The mandate and various info about the work of this group can be found here: https://twiki.cern.ch/twiki/bin/view/LCG/GSSD
    The mailing list is: storage-classes-wg@cern.ch
    Speaker: Flavia Donno
  • <big> BDII present deployment scenario in all regions </big> 10m
  • 16:55 → 17:00
    OSG Items 5m
    Item 1
  • 17:00 → 17:45
    WLCG Items 45m
  • 17:45 → 17:55
    Review of action items 10m
    more information
  • 17:55 → 18:00
    AOB 5m