Ceph/CVMFS/Filer Service Meeting

Europe/Zurich
600/R-001 (CERN)

600/R-001

CERN

4
Show room on map
Description

Zoom: Ceph Zoom

    • 14:00 14:15
      CVMFS 15m
      Speakers: Enrico Bocchi (CERN) , Fabrizio Furano (CERN)

      Enrico:

      • Catalog splits on ams.cern.ch
      • Clean-up of old RMs on SLC6 and volumes
      • Migration of sft-nightlies this Thu @14 (OTG0060030)
      • Waiting for bugfix on catalogs for lhcbdev.cern.ch
      • Preparing for migration of cms and cms-ib -- last two on Cinder
       
    • 14:15 14:30
      Ceph: Operations 15m
      • Incidents, Requests, Capacity Planning 5m
        Speaker: Dan van der Ster (CERN)
      • Cluster Upgrades, Migrations 5m
        Speaker: Theofilos Mouratidis (CERN)
        • New koji tag "ceph8", e.g.
          • http://linuxsoft.cern.ch/internal/repos/ceph8-stable/x86_64/os/Packages/
        • Built 14.2.11-2 there, along with new smartmontools-7.1
        • Recap:
          • ceph7/ceph8 are for our servers (hg_ceph and module ceph)
          • ceph-client7/ceph-client8 are for the cephfs clients (module cephfs)

        14.2.13 is being built

      • Hardware Repairs 5m
        Speaker: Julien Collet (CERN)
      • Puppet and Tools 5m
    • 14:30 14:45
      Ceph: Projects, News, Other 15m
      • Kopano/Dovecot 5m
        Speaker: Dan van der Ster (CERN)
      • REVA/CephFS 5m
        Speaker: Theofilos Mouratidis (CERN)

        1 PR to be done

        • One commit that removes the mtime changing rctime
          • Tests still on the making
        • One commit that makes rctime modifiable so it can be recovered
          • Tests still on the making
    • 14:45 14:55
      S3 10m
      Speakers: Julien Collet (CERN) , Roberto Valverde Cameselle (CERN)

      Giuliano

      • CEPH-993: rgw_max_concurrent_requests:
        • Set up a VM (warp-xl-benchmark) to issue concurrent get requests to one of the nethub cluster rgw
        • For each value of --concurrent ranging between 500 and 1500, the option was tuned to the following values {1024,512,256,128}
        • What is observed is that, indeed, each time to you divide by two this number, you roughly divide by two the RSS (except 512/1024 that show roughly same memory footprint,to be investigated? Might be the VM that cannot really issue all these requests at the same time.)
        • One could relatively clearly see that the throttling was kicking in since warp was reporting: "warp: <ERROR> download error: Error response code SlowDown."
        • TL; DR
          • I think it might pay off to tune it to 512 and see if this needs further tuning afterwards.
    • 14:55 15:05
      Filer/CephFS 10m
      Speakers: Dan van der Ster (CERN) , Theofilos Mouratidis (CERN)
      • Filer-carbon ran out of disk due to a user pushing 1M new metrics. (Each metric needs 300kB).
        • I deleted the metrics and added space monitoring to /var/lib/carbon
        • Also upgrade grafana and go-carbon to the latest versions
    • 15:05 15:10
      JIRA SCRUB 5m