US ATLAS Computing Integration and Operations

→ US/Eastern
Description
Notes and other material available in the US ATLAS Integration Program Twiki
    • 13:00 → 13:15
      Top of the Meeting 15m
      Speakers: Eric Christian Lancon (CEA/IRFU,Centre d'etude de Saclay Gif-sur-Yvette (FR)), Robert William Gardner Jr (University of Chicago (US))

      Apologies: Kaushik

       

      MWT2: PO's out for both RBT and FY16.  5376 TB, 45k HS06

      AGLT2: PO's issued last week.  UM - the order was submitted but had to be corrected.  Still in the submit phase. Goal is to get it operational by end of August.  ~25 kHS, ~3.5 PB

      NET2: SOW and 1030 to Jim.  Order is in the system and approved.  1536 TB, 24 UTA style worker nodes, and a switch.

      SWT2:  purchase request has been submitted to UTA - they are waiting for paperwork to come back from Columbia before they release. We are buying 2.91TB (usable) of storage and ~27K HS06 at UTA, and 500TB (usable) at OU, will be put into the OSCER cluster.

       

       

    • 13:25 → 13:35
      Capacity News: Procurements & Retirements 10m
    • 13:35 → 13:45
      Production 10m
      Speaker: Mark Sosebee (University of Texas at Arlington (US))
    • 13:45 → 13:50
      Data Management 5m
      Speaker: Armen Vartapetian (University of Texas at Arlington (US))
    • 13:50 → 13:55
      Data transfers 5m
      Speaker: Hironori Ito (Brookhaven National Laboratory (US))
    • 13:55 → 14:00
      Networks 5m
      Speaker: Dr Shawn McKee (University of Michigan ATLAS Group)
    • 14:00 → 14:05
      FAX and Xrootd Caching 5m
      Speakers: Andrew Bohdan Hanushevsky (SLAC National Accelerator Laboratory (US)), Andrew Hanushevsky, Andrew Hanushevsky (STANFORD LINEAR ACCELERATOR CENTER), Ilija Vukotic (University of Chicago (US)), Wei Yang (SLAC National Accelerator Laboratory (US))
    • 14:25 → 15:25
      Site Reports
      • 14:25
        BNL 5m
        Speaker: Eric Christian Lancon (CEA/IRFU,Centre d'etude de Saclay Gif-sur-Yvette (FR))
      • 14:30
        AGLT2 5m
        Speakers: Robert Ball (University of Michigan (US)), Dr Shawn McKee (University of Michigan ATLAS Group)

        One of the 2 WAN circuits from AGLT2 was found to be down.  A failed optic was the cause.  A replacement brought the circuit up again and returned our WAN to 80Gb.

        The LAN between the UM and MSU sites is also a dual-40Gb pair, and one of these is showing increasing losses.  This is still under diagnosis.

        The POs for hardware at MSU have been submitted to Dell.  Those for UM are still working their way through purchasing, but we expect an imminent placement for them.

         

      • 14:35
        MWT2 5m
        Speakers: David Lesny (Univ. Illinois at Urbana-Champaign (US)), Lincoln Bryant (University of Chicago (US))

        Site  is running well

        Illinois is currently down for a PM

        FY16 Purchases (~49K HS06 via 4224 logical cores, 5.25PB dCache disk)

        • UChicago
          • Compute Nodes - 48 R630 nodes
            • E5-2640 v4 (10/20 cores), 40 Logical Cores/node
            • 1920 Logical Cores @ ~11.57HS06/lc = ~22.2K HS06
            • 128GB/node (3.2GB/LC)
            • 2x800GB SDD drives (~30GB/LC)
          • Storage Servers - 7 R730/xd nodes
            • E5-2680 v4 (14/28 cores), 56 Logical Cores/nodes
            • 256GB/node
            • 2x500GB SAS drive for system
          • Storage systems - 7 x (3460 + 3060e)
            • 840 x 8TB drives
            • About 5.25PB usable space
            • MWT2 dCache ~9PB
          • Other R730/R630 for hypervisor, perfSonar, analytics and dCache name server nodes
          • Juniper switch and blades need to be ordered
            • 10Gb and 40Gb blades for Juniper 9208
              • Storage servers at 4x10Gb
              • Top of rack at 2x10Gb and 40Gb
            • Juniper 3400 top of rack for 1Gb nodes
            • Juniper 4550 top of rack for 10Gb nodes (R630)
        • Indiana
          • Compute Nodes - 24 R630 nodes
            • E5-2640 v4 (10/20 cores), 40 Logical Cores/node
            • 960 Logical Cores @ ~11.57HS06/lc = ~11.1K HS06
            • 128GB/node (3.2GB/LC)
            • 2x800GB SDD drives (~30GB/LC)
        • Illinois
          • Nodes must conform to Illinois Campus Cluster configuration and pricing
          • Dell confirmed the order
          • Network switches ordered (Juniper QFX5200)
            • Electricians are no available due to summer projects on campus
            • Racks and installation will be delayed until fall
            • Nodes should be delivered and online by end of the month
          • 6 x C6320 chassis (4 nodes per chassis, 24 two socket nodes)
            • E5-2680 v4 (56 logical cores/node with HT enabled)
            • 11.57 HS06/LC
            • 1344 Logical Cores
            • Adds 15,550 HS06
            • 256GB (4.5GB/LC)
            • 10Gb to each node

         

        Site updates

        • OSG 3.3.14
        • Condor 8.4.8
        • CVMFS 2.2.3
      • 14:40
        NET2 5m
        Speaker: Prof. Saul Youssef (Boston University (US))
      • 14:45
        SWT2-OU 5m
        Speaker: Dr Horst Severini (University of Oklahoma (US))

        - No operational problems, all sites running smoothly

        - Shut down old OSCER cluster (Boomer) and migrated to new Schooner cluster (in maintenance today)

        - Still issues with MCORE jobs (AthMpEvtLoopMgr), need to open a ticket on that

        - About to order 5 T630s with 100 TB usable each, plus SE frontend and xrootd redirector, and required additional hardware, for new OU_OSCER_ATLAS SE

         

      • 14:50
        SWT2-UTA 5m
        Speaker: Patrick Mcguigan (University of Texas at Arlington (US))
      • 14:55
        WT2 5m
        Speaker: Wei Yang (SLAC National Accelerator Laboratory (US))
    • 15:25 → 15:30
      AOB 5m