Go to the Indico Home Page Oldest event Older event Up to category Newer event Newest event Printable version
 
Focus on:  Hide Contributions
  • Compact style
  • Indico style
  • Indico style - inline minutes
  • Indico style - numbered
  • Indico style - numbered + minutes
  • Indico Weeks View
    • Switch to the management area of this event
    Login

    RAL Tier1 Experiments Liaison Meeting

    → Europe/London
    Access Grid (RAL R89)

    Access Grid

    RAL R89
    RAL Tier1 Experiments Liaison Meeting
    Zoom Meeting ID
    66811541532
    Host
    Alastair Dewhurst
    Useful links
    Join via phone
    Zoom URL
      • 14:00 → 14:01
        Major Incidents Changes 1m
      • 14:01 → 14:02
        Summary of Operational Status and Issues 1m
        Speakers: Brian Davies (Lancaster University (GB)), Darren Moore (Science and Technology Facilities Council STFC (GB)), Kieran Howlett (STFC RAL)
        Weekly Report 04 October 2023 .docx
        Weekly Report 04 October 2023 .pdf
      • 14:02 → 14:03
        GGUS /RT Tickets 1m

        https://tinyurl.com/T1-GGUS-Open
        https://tinyurl.com/T1-GGUS-Closed

      • 14:04 → 14:05
        Site Availability 1m

        https://lcgwww.gridpp.rl.ac.uk/utils/availchart/

        https://cms-site-readiness.web.cern.ch/cms-site-readiness/SiteReadiness/HTML/SiteReadinessReport.html#T1_UK_RAL

        http://hammercloud.cern.ch/hc/app/atlas/siteoverview/?site=RAL-LCG2&startTime=2020-01-29&endTime=2020-02-06&templateType=isGolden

      • 14:05 → 14:06
        Experiment Operational Issues 1m
      • 14:15 → 14:16
        VO Liaison CMS 1m
        Speaker: Katy Ellis (Science and Technology Facilities Council STFC (GB))
      • 14:16 → 14:17
        VO-Liaison ATLAS 1m
        Speakers: James William Walder (Science and Technology Facilities Council STFC (GB)), Jyoti Prakash Biswal (Rutherford Appleton Laboratory)
        • GGUS-163297:
          • RAL-LCG2: deletion errors; in progress.
          • After the brief downtime last week on Echo, the rate appeared to improve slightly. 
          • Also tried to move between different protocols and across a couple of DNS aliases => root protocol produces far fewer timeouts than davs.
          • Need some discussion with XRootD developers!
          • Likely to move RAL back to root for deletions.
          • Hopefully  the ticket is likely to be resolved once RAL is back to its pledge levels, and continue to understand this behaviour.

         

        • GGUS-163423:
          • RAL-LCG2_MCTAPE: staging failures (max-time-in-queue timeout); solved.
          • Not staging but files transfer from tape to RAL disk --> staged on to the buffer efficiently, but FTS copying was delayed. 
          • The reason for the time to copy was that there were many many files wanting to write to RAL disk ,and there was contention for the number of active links.
          • Improved situation over the last weekend. 

         

        • RAL staging blocks some validation:
          • minbias on tape, secondaries not staged by DC.
          • Removed RAL tape resident dataset from 2 containers (hi s4102,lo s4105), in order to use 2nd dataset with replica on other tape sites. Staging of these worked fine.
            • Can be put back later
          • Cloned slice 1 to extend with another 200k events.

         

        • RAL inbound backlog:
          • 60k staged files not moving to disk at all - 0 active. INC3612840 found different priorities within link caused the problem. Set all to the same priority, then it worked.

         

        • Anywhere to RAL disk FTS limit is set to only 40. To be changed back to default 100. Already done?

         

        • Link to the presentation on DAOD-on-demand test (during the ongoing S&C week).
          • Paired tests are to be performed at RAL. 

         

        • Observation of  the swapping issue ongoing. 
      • 14:20 → 14:21
        VO Liaison LHCb 1m
        Speaker: Alexander Rogovskiy (Rutherford Appleton Laboratory)
        63172f2a44dbd1d81c6b977790a428b6.png

        Tickets:

        1. CVMFS update issue.
          This is causing serious troubles for LHCb jobs (see plots), so Stratum-1 shutdown is desireable.
        2. Failed uploads from RAL WN.
          Since the DNS poisoning had been put in place, there were almost no issues with WN->ECHO transfers
          "External" transfers looks also more or less OK, though a little bit worse then WN->ECHO ones (e.g. last night there was a spike of failed transfers from other sites).
        3. Redirector setup
          No updates
        4. Slow stats/checksums
          No updates
           

        Operational issues:

        1. LHCb has very little jobs to run
          This is not our fault, all sites are affected.
        2. Vector read performance was not ideal the week before last (see plot attached). Most probably due to the roll-back of the "No prefetch" config.
      • 14:25 → 14:28
        VO Liaison LSST 3m
        Speaker: Timothy John Noble (Science and Technology Facilities Council STFC (GB))
      • 14:30 → 14:31
        VO Liaison Others 1m
      • 14:31 → 14:32
        AOB 1m
      • 14:32 → 14:33
        Any other Business 1m
        Speakers: Brian Davies (Lancaster University (GB)), Darren Moore (Science and Technology Facilities Council STFC (GB))