Ceph/CVMFS/Filer Service Meeting

600/R-001 (CERN)



Show room on map
Videoconference Rooms
Dan van der Ster
Auto-join URL
Useful links
Phone numbers
    • 09:30 09:45
      CVMFS 15m
      Speakers: Enrico Bocchi (CERN), Fabrizio Furano (CERN)


      • Migration of lhcb.cern.ch and alice.cern.ch
        • Last two machines affected by cloud HW refresh
        • lxcvmfs67, lxcvmfs68: 4y 10 months
      • Test machines for SFT repos (sft, sft-nightlies) ready and delivered
      • Issue with site caches (ca-proxy) and ilc.desy.de/.cvmfspublished "TCP_HIT_ABORTED/000":
        • Seems to be somehow related to collapsed_forwarding (the issue disappears by disabling it)
        • Main developer of frontier flavor of squid did not manage to reproduce the issue, though he acknowledges there was a recently-introduced bug with collapsed_forwarding and If-Modified-Since (http://frontier.cern.ch/dist/rpms/frontier-squidRELEASE_NOTES)
        • Version with the fix still shows the problem
        • Intermittent and not completely understood
        • Plan to get in touch with Stratum administrators at Desy
      • Issue with Stratum 1 caches (frontN) "TCP_REFRESH_FAIL_OLD/200":
    • 09:45 10:00
      Ceph: Operations 15m
      • Cluster Upgrades, Migrations 5m
        Speaker: Theofilos Mouratidis (CERN)

        v14.2.10 has been released. We start the review and testing procedures before upgrading important clusters.

      • Hardware Repairs 5m
        Speaker: Julien Collet (CERN)
      • Incidents, Requests, Capacity Planning 5m
        Speaker: Dan van der Ster (CERN)
        • ITUM: vault-100 and vault-500 block storage announced.
      • Puppet and Tools 5m
    • 10:00 10:15
      Ceph: Projects, News, Other 15m
      • Kopano/Dovecot 5m
        Speaker: Dan van der Ster (CERN)
      • REVA/CephFS 5m
        Speaker: Theofilos Mouratidis (CERN)
        • Basic cephfs module based on localfs done, changed etag calculation to just (r)ctime+inode
        • rctime will get updated to all parent dirs within 4-5 seconds regardless of load
        • Smashbox test this week to benchmark Etags
        • on flax we have https://its.cern.ch/jira/browse/CEPH-925 -- rctime in the future. need to test a nautilus cluster to see if this happens there too.
    • 10:15 10:25
      S3 10m
      Speakers: Julien Collet (CERN), Roberto Valverde Cameselle (CERN)
      • INC2478158: jakob had to cancel his migration of the cervm repo website to s3-websites because he needs https to work, but our certs don't cover the correct names.
      • INC2473417: root.cern.ch has a default index issue. tracker opened at https://tracker.ceph.com/issues/46241

      theo: https://github.com/ceph/ceph/pull/35792



      • S3 Logs:
        • Log-filtering job was indeed working since the logs of 3w ago don't have the fields listed under `drop_field`. However, more recent logs show these new fields.
        • Will be clarified this week
      • S3 Project:
        • Last week, multi-zones have been set up as with zone replication
        • Set up of the es-sync module
        • Need to check for the pubsub fix in 14.2.10
      • Accounting:
        • VM fired up with all the tools, puppetization in progress
    • 10:25 10:35
      Filer/CephFS 10m
      Speakers: Dan van der Ster (CERN), Theofilos Mouratidis (CERN)
    • 10:35 10:40
      AOB 5m