Site admininistration

Middleware

General view

  • CPU
    • CentOS7 with singularity (for 2 years)
    • 5000 logical CPUs
      • actually half used for 8 core CMS jobs
  • storage
    • 1.5 PB
    • 1.2 PB pledged to CMS
  • dual stack

CE and worker nodes

  • OS: CentOS 7
  • Batch:
    • Condor
    • Slurm (MPI for HPC)
  • CE: Arc 5 and Arc 6 (almost)

Storage in production

  • 1.5 PB
  • DOME 1.2
    • since March
    • first in WLCG
      • 1.11 and 1.12 debugged at Brunel in frantic 6 weeks
  • production storage p8ssing TPC smoking test since March
  • production storage in TPC stress test since March
  • File systems
    • XFS (mostly)
    • BtrFS (2 unit, 340 TB)
    • ZFS (1 unit, 40 TB)

Network Status

Network resources available

  • Present link: 20 Gbps
  • upcoming: 40 Gbps (depending on Janet delivery)
  • everything dual stack for 5 years, except…
  • 1 CE on pure IPv6
    • used by Atlas, LHCB, LSST (?!)
  • experimental DTN
    • DTN with OAuth
    • gridftp
    • davs
    • QUIC

Network usage (90 days)

Transfer to UK sites (90 days)

Transfer to UK sites (90 days)

Network usage (IPv6)

Transfer to UK sites (90 days)

Transfer to UK sites (90 days)

Network snapshot (on ultimatum)

Transfer to UK sites (90 days)

Transfer to UK sites (90 days)

Future

  • DPM/DOME: what next?
  • Impact of moving
    • Gaining expertise
    • downtime (how long?)
      • 2010 again?