Go to the Indico Home Page Oldest event Older event Up to category Newer event Newest event Printable version
  • Compact style
  • Indico style
  • Indico style - inline minutes
  • Indico style - numbered
  • Indico style - numbered + minutes
  • Indico Weeks View
    • Switch to the management area of this event
    Login

    ATLAS UK Cloud Support

    → Europe/London
    Vidyo

    Vidyo
    Tim Adye (Science and Technology Facilities Council STFC (GB)), James William Walder (Science and Technology Facilities Council STFC (GB))
    Hide

    ● Outstanding tickets

    • GGUS #146704 Lancaster, Matt: running out of room. Does it help to make make full disk read-only in DPM? Can still delete.
      Sam said it is safe as long as the JSON file is used for space tokens, rather than SRM. Can still delete files in DPM "read-only" mode. So OK for Lancaster.
      Sam suggests to keep 10% overhead on top of ATLAS 10% free to allow for site shuffling.
      Lancaster currently have 2.07 PB with 200 TB free. The disk has a lot of secondary data, so ATLAS should be fine with reducing quota. Matt will reduce the quota to trigger deletion.
    • GGUS #146687 Sheffield in progress
    • GGUS #146651 RAL, James: Singularity (running inside Condor docker container) needs new version. Alastair deciding which version. In progress.
    • GGUS #146619 RALPP currently in downtime for upgrade to ARC-CE. Will look at this issue when that's done.
    • GGUS #146588 RAL (James) in the process of upgrading Squids.
    • GGUS #146525 Sheffield in progress. Elena is rebuilding some machines.
    • GGUS #146523 Manchester still in investigation, but this issue is not specific to Manchester.
    • GGUS #146374 Sheffield same as #146525.
    • GGUS #146159 Glasgow disk now fully dead while getting unique files off disk. Everything else should be declared lost.
    • GGUS #145688 Alessandra said last week that they should make progress soon.
    • GGUS #145510 SSD worker nodes give better performance, but still lower efficiency than expected. Investigations proceeding.
    • GGUS #144759 Glasgow: Gareth looked at original problem. Load didn't seem that unexpected.
      Two issues: 1) Router issue that Gareth will discuss with local IT. 2) One Squid sometimes breaks and fails over to CERN. This can't be fixed at the moment.
      Gareth will put an update in the ticket summarising the situation.
    • GGUS #142329 Sussex: No news from Patrick (via Peter)

    ● CPU

    • Matt: Lancaster drop due to the full disk
    • Rob: ECDF everything should be working this morning.
    • James: Problem with HammerCloud interaction with Grand Unified PanDA queues that stopped queue coming online. Will ask HammerCloud to fix.

    ● Other new issues

    Durham, Paul Clark:

    • Looking at the Squid issue. James will send Paul the PanDA log files.
    • Wants to move away from SRM. Agreed to sort out Squid issue first, then look into decommissioning SRM.

    ● Glasgow Ceph storage

    Problem with XRootD writing to new gateway node; worked on the old one.

    ● Grand Unified queues

    Only Sheffield still to complete.

    ● News round-table

    • Dan: NTR
    • Gareth: NTR
    • James: NTR
    • Matt: NTR
    • Paul: NTR. Will try to connect in future.
    • Peter: NTR
    • Rob: NTR
    • Sam: NTR
    • Tim: NTR
    • Vip: had to leave early for another meeting.
    There are minutes attached to this event. Show them.
      • 10:00 → 10:20
        Status 20m
        • Outstanding tickets 10m
          Open ATLAS UK GGUS tickets
          • GGUS #146704 Lancaster, Matt: running out of room. Does it help to make make full disk read-only in DPM? Can still delete.
            Sam said it is safe as long as the JSON file is used for space tokens, rather than SRM. Can still delete files in DPM "read-only" mode. So OK for Lancaster.
            Sam suggests to keep 10% overhead on top of ATLAS 10% free to allow for site shuffling.
            Lancaster currently have 2.07 PB with 200 TB free. The disk has a lot of secondary data, so ATLAS should be fine with reducing quota. Matt will reduce the quota to trigger deletion.
          • GGUS #146687 Sheffield in progress
          • GGUS #146651 RAL, James: Singularity (running inside Condor docker container) needs new version. Alastair deciding which version. In progress.
          • GGUS #146619 RALPP currently in downtime for upgrade to ARC-CE. Will look at this issue when that's done.
          • GGUS #146588 RAL (James) in the process of upgrading Squids.
          • GGUS #146525 Sheffield in progress. Elena is rebuilding some machines.
          • GGUS #146523 Manchester still in investigation, but this issue is not specific to Manchester.
          • GGUS #146374 Sheffield same as #146525.
          • GGUS #146159 Glasgow disk now fully dead while getting unique files off disk. Everything else should be declared lost.
          • GGUS #145688 Alessandra said last week that they should make progress soon.
          • GGUS #145510 SSD worker nodes give better performance, but still lower efficiency than expected. Investigations proceeding.
          • GGUS #144759 Glasgow: Gareth looked at original problem. Load didn't seem that unexpected.
            Two issues: 1) Router issue that Gareth will discuss with local IT. 2) One Squid sometimes breaks and fails over to CERN. This can't be fixed at the moment.
            Gareth will put an update in the ticket summarising the situation.
          • GGUS #142329 Sussex: No news from Patrick (via Peter)
        • CPU 5m
          UK Cloud jobs over last week
          • Matt: Lancaster drop due to the full disk
          • Rob: ECDF everything should be working this morning.
          • James: Problem with HammerCloud interaction with Grand Unified PanDA queues that stopped queue coming online. Will ask HammerCloud to fix.
        • Other new issues 5m

          Frontier cache at Durham
          squid.dur.scotgrid.ac.uk:3128

          Durham, Paul Clark:

          • Looking at the Squid issue. James will send Paul the PanDA log files.
          • Wants to move away from SRM. Agreed to sort out Squid issue first, then look into decommissioning SRM.
      • 10:20 → 10:40
        Ongoing issues 20m
      • 10:40 → 10:50
        News round-table 10m
        • Dan: NTR
        • Gareth: NTR
        • James: NTR
        • Matt: NTR
        • Paul: NTR. Will try to connect in future.
        • Peter: NTR
        • Rob: NTR
        • Sam: NTR
        • Tim: NTR
        • Vip: had to leave early for another meeting.
      • 10:50 → 11:00
        AOB 10m