Ceph/CVMFS/Filer Service Meeting

600/R-001 (CERN)



Show room on map
    • 2:00 PM 2:05 PM
      CVMFS 5m
      Speaker: Enrico Bocchi (CERN)
    • 2:05 PM 2:10 PM
      Ceph Upstream News 5m

      Releases, Tickets, Testing, Board, ...

      Speaker: Dan van der Ster (CERN)
      • v14.2.4 is released, and it is quite stable but has one known MDS bug (https://tracker.ceph.com/issues/41935) caused by an incomplete backport. It's a good time to plan an upgrade for ceph/erin.
      • Recently the "mimic missing slow OSD ops" bug has been fixed (https://tracker.ceph.com/issues/40993)
        • Devs have noticed many more warnings in qa testing since this was merged -- maybe the fix is incomplete (or the fix has revealed other problems that had been masked by the bug).
    • 2:10 PM 2:15 PM
      Ceph Backends & Block Storage 5m

      Cluster upgrades, capacity changes, rebalancing, ...
      News from OpenStack block storage.

      Speaker: Theofilos Mouratidis (National and Kapodistrian University of Athens (GR))


      • On ceph/beesly all balancing/disk replacement is paused due to filestore splitting: https://its.cern.ch/jira/projects/CEPH/issues/CEPH-750
        • Background: Filestore stores objects in XFS, and creates subdirs onces the number of objects in a directory reaches some threshold. This "splitting" adds some small latency whenever it is triggered, so in the past we have worked around this by raising the threshold. Now the threshold is so high, that when the split is triggered it causes a hang of 10s of seconds.
          • I have started a campaign to split the PGs into smaller directories -- this is done offline, while the OSD is stopped, to prevent any slow requests.
          • Should be done by end of this week, after which we can resume balancing/etc...
    • 2:15 PM 2:20 PM
      Ceph Disk Management 5m

      OSD Replacements, Liaison with CF, Failure Predictions

      Speaker: Julien Collet (CERN)
    • 2:20 PM 2:25 PM
      S3 5m

      Ops, Use-cases (backup, DB), ...

      Speakers: Julien Collet (CERN) , Roberto Valverde Cameselle (Universidad de Oviedo (ES))
    • 2:25 PM 2:30 PM
      CephFS/HPC/FILER/Manila 5m

      Filer Migration, CephFS/Manila, HPC status and plans.

      Speakers: Dan van der Ster (CERN) , Pablo Llopis Sanmillan (CERN)
      • LSF Filers (itnfs22b, itnfs24b) have been decommissioned following a request from the LSF service.
    • 2:30 PM 2:35 PM
      HyperConverged 5m
      Speakers: Jose Castro Leon (CERN) , Julien Collet (CERN) , Roberto Valverde Cameselle (Universidad de Oviedo (ES))
    • 2:35 PM 2:40 PM
      Monitoring 5m


      • Updated prometheus puppet module to match puppet forge version. 
    • 2:40 PM 2:45 PM
      AOB 5m