ROC Managers' Meeting

→ Europe/Zurich
CERN

CERN

Nick Thackray
Description
Actions, https://edms.cern.ch/document/753089 Minutes, https://edms.cern.ch/document/829338 This meeting is 10:00 to 11:30 UTC (11:00 to 12:30 Swiss local time). Phone number is: +41 22 767 6000 Access code is: 0147097 Or click here: https://audioconf.cern.ch/call/0147097 The conference call opens 5 minutes before the meeting starts.
Recording of the meeting
    • 11:00 → 12:30
      ROC Managers' Meeting
      • 11:00
        Admin matters 10m
      • Quarterly Report
      • Meetings
      • Milestones
      • Deliverables
      • Moderators
      more information
  • 11:10
    GGUS reports 15m
    Speaker: Maria Dimou
    • TPM Monitoring
      more information
    • GGUS ticket escalation report
      more information
  • 11:25
    Upgrading of Top Level BDIIS 5m
    With the release of version 3.9.0-1 of the BDII, a very significant improvement was made to the performance of the service (through the use of indices). However, it is suspected that many top level BDIIs have not been upgraded to the most recent version. So, with reference to the next agenda item, we need to make sure that the "official" top level BDIIs are at the latest version.
  • 11:30
    Organization of Top level BDIIs 15m
    At the moment we have many top-level BDIIs: GOOD
    However, many of them are working poorly or not at all: BAD

    The list of BDIIs with problems is here:
    • Response time >10s
      • lcg00005.grid.sinica.edu.tw
      • lcg00126.grid.sinica.edu.tw
      • lcg007.ihep.ac.cn
      • lcg2bdii.tifrgrid.res.in
      • lcg-bdii01.icepp.jp
      • PAKBDII.pakgrid.org.pk
      • pcncp24.ncp.edu.pk
      • rb1.egee.fr.cgg.com
      • rls04.cc.kek.jp
      • srmse.inrne.bas.bg
      • t-op18.grid.sinica.edu.tw
      • top-bdii.triumf.ca
      • zeus25.cyf-kr.edu.pl
    • Content problems
      • lcg-01.hpl.hp.com
      • lcg03.physics.carleton
      • lcg-bdii.gsi.de
      • neptun.switch.ch
      • pps-bdii.cern.ch
      • skurut67-4.cesnet.cz
      • ui.ulakbim.gov.tr
    To improve the situation, the proposal is the following:
    1. Each ROC makes sure the region runs a load balanced BDII with 2+ nodes
    2. Sites with more than, say 1000 CPUs set up a local top-level BDII for local usage
    3. CERN provides a default load balanced BDII to be used when the "local" services are not available
    4. SAM tests these production BDIIs. Sites can use other BDIIs at their own risk (no support)
    5. In addition:
      • We ask for SAM/Gstat to monitor the top level BDIIs, with the intention alarms being raised for the CODs
      • We ask that the middleware be enhanced so that job submission can either be configured with a list of BDIIs or can discover BDIIs (service discovery)
  • 11:45
    How to deal with high loads on RBs/WMSs caused by users? 15m
    Use case: A user or VO wishes to submit thousands of jobs to the grid. They have 3 RBs/WMSs available to them but two of the RBs/WMSs are "out of action" for some reason (down; mis-configured; etc.) so the user submits all their jobs through the remaining good RBs/WMSs. This introduces an unacceptable load on the RBs/WMSs and the site raises a ticket for the VO or even bans the user. Q: How should the grid deal with this?
  • 12:00
    The Quarterly Report 10m
  • 12:10
    Review of action items 10m
  • 12:20
    AOB 5m
    • Departure of David Kant from Rutherford (+ how items related to the accounting and monitoring work will be dealt with)
    • Updated draft version of SLD to circulate to sites for comments. EDMS location: https://edms.cern.ch/document/860386/0.5