RAL Tier1 Experiments Liaison Meeting

→ Europe/London
Access Grid (RAL R89)

Access Grid

RAL R89

Zoom Meeting ID
66811541532
Host
Alastair Dewhurst
Useful links
Join via phone
Zoom URL
    • 13:30 → 13:34
      Site Operations 4m
    • 13:34 → 13:35
      Experiment Operational Issues 1m
    • 13:35 → 13:40
      ATLAS Operations Report 5m
      Speakers: Brij Kishor Jashal (Rutherford Appleton Laboratory), Jyoti Prakash Biswal (Rutherford Appleton Laboratory)
    • 13:40 → 13:45
      CMS Operations Report 5m
      Speaker: Katy Ellis (Science and Technology Facilities Council STFC (GB))
    • 13:45 → 13:50
      LHCb Operations Report 5m
      Speaker: Alexander Rogovskiy (Rutherford Appleton Laboratory)

      Operational issues:

      • Failed transfers to ECHO last week (GGUS ticket)
        • Too "aggressive" data migration overloaded 2019 SN gen, fixed by reducing the migration "priority"
      • Failed user jobs at RAL (GGUS ticket)
        • The fix passed CC last week and is deployed to preprod farm, looks OK so far
      • Problem with transfers to/from Beijing to ECHO last week
        • the usual SHA-1 issue after the redirector upgrade to EL9
      • A spike of failed uploads from outside RAL to ECHO yesterday morning
        • Seems to be a DNS issue
          • Confirmed to be a DI issue


      News:

      • VO-box migrated to proxmox last week
    • 13:50 → 13:55
      ALICE Operations Report 5m
      Speaker: Alexander Rogovskiy (Rutherford Appleton Laboratory)
      • Data export to RAL seems to be over.
      • VO-box migration is planned for this Thursday
      • Xrootd port on alice.echo.stfc.ac.uk  is not contactable via IPv6 from outside of RAL (e.g. lxplus)
        • There was an old DI ticket IT-25-43996, what's its status?
          • Agreed to postpone the fix until the migration to proxmox is finished, since the host is going to end up in a new network
    • 13:55 → 14:00
      LSST Operations Report 5m
      Speakers: Mathew Sims, Timothy Noble (Science and Technology Facilities Council STFC (GB))
      • DP2_prep
        • Still awaiting Database to be moved to allow the jobs to be run
          • Fabric have now started the creation of the VMs as of this morning
          • Still have to snapshot and move database over
        • Mislabelling of datasets at Observatory has caused some issues with dataset
          • Issues identified and updates sent to sites over the weekend
            • Caused FTS issues which then led to race conditions and deleted files at sites
          • Working on getting this sorted at RAL - fix workflow, still being identified at all sites
      • FTS
        • LSST has been using LCGFTS, and some people within LSST have thought they would use one deployed at SLAC
          • Issues with the SLAC deployment has meant they have reverted to using RAL
          • LCGFTS has been scaled down due to lack of use and project funding
          • LSST:UK considering funding for FTS to ensure a service is provided that they can use reliably 
      • VOBox
        • Migration underway but CVMFS not playing ball
          • Issue being resolved
      • Few Jobs run at RAL this week or last
      • Data transfers to RAL of 14 TB over the last 2 weeks -  now up to 670 TB on Echo
         
    • 14:00 → 14:01
      Tier-1 Projects 1m
    • 14:02 → 14:07
      Anatares Upgrade 5m
      Speakers: George Patargias, Thomas Byrne
    • 14:08 → 14:13
      XRootD Development 5m
      Speakers: Alexander Rogovskiy (Rutherford Appleton Laboratory), Jyothish Thomas (STFC)

      streamed checksum bug has been fixed, but the performance went down to a point where functional checks are failing. Further investigation in progress
      Issues on gateways due to storage slowdown & DNS DI outage

    • 14:14 → 14:19
      Utilizing GPUs 5m
      Speakers: Jyoti Prakash Biswal (Rutherford Appleton Laboratory), Thomas Birkett
    • 14:20 → 14:25
      SSD Storage Evaluation 5m
    • 14:25 → 14:26
      AOB 1m
    • 14:27 → 14:36
      Summary of Operational Status and Issues 9m
      Speakers: Brian Davies (Science and Technology Facilities Council STFC (GB)), Darren Moore
    • 14:45 → 14:50
      Any other Business 5m
      Speakers: Brian Davies (Science and Technology Facilities Council STFC (GB)), Darren Moore