US ATLAS Computing Integration and Operations

→ US/Eastern
Description
Notes and other material available in the US ATLAS Integration Program Twiki
    • 13:00 → 13:15
      Top of the Meeting 15m
      Speakers: Eric Christian Lancon (CEA/IRFU,Centre d'etude de Saclay Gif-sur-Yvette (FR)), Robert William Gardner Jr (University of Chicago (US))
    • 13:15 → 13:20
      ADC news and issues 5m
      Speakers: Robert Ball (University of Michigan (US)), Wei Yang (SLAC National Accelerator Laboratory (US))

      Via email from Bob:

      ADC coverage over break is detailed here: https://indico.cern.ch/event/595726/attachments/1391128/2119183/go 

      ~760M events in the production queue for the break, 240M evgen, 400M simulation, 120M digit+reco, which should be enough to last until January.

      Enough group production for about 1.5mo of MC running. 

      Group Processing:  Derivation jobs now submitted with 20GB (MC) and 30GB (Data) as input per job, but these are MCORE now so should not be a big impact on sites.

      The ADC is collecting information on useful ATLAS dashboards.  Some are going away, and we may want to replicate some functionality, some are new/prototypes.  If you have a favorite that you rely upon, please enter the information in the following Google doc.  If you cannot edit the doc, please send the information to ball@umich.edu and Bob will enter the information.

      https://docs.google.com/document/d/1R0mT7bOh-pCIDY8pUvGA6VAfXKmgvXFbGyTVbVM9EcI/edit

      bob

       

    • 13:20 → 13:30
      Production 10m
      Speaker: Mark Sosebee (University of Texas at Arlington (US))
    • 13:30 → 13:35
      Data Management 5m
      Speaker: Armen Vartapetian (University of Texas at Arlington (US))
    • 13:35 → 13:40
      Data transfers 5m
      Speaker: Hironori Ito (Brookhaven National Laboratory (US))
    • 13:40 → 13:45
      Networks 5m
      Speaker: Dr Shawn McKee (University of Michigan ATLAS Group)
    • 13:45 → 13:50
      FAX and Xrootd Caching 5m
      Speakers: Andrew Bohdan Hanushevsky (SLAC National Accelerator Laboratory (US)), Andrew Hanushevsky, Andrew Hanushevsky (STANFORD LINEAR ACCELERATOR CENTER), Ilija Vukotic (University of Chicago (US)), Wei Yang (SLAC National Accelerator Laboratory (US))
    • 13:50 → 14:10
      Site movers 20m
      Speaker: Wei Yang (SLAC National Accelerator Laboratory (US))
    • 14:10 → 14:30
      OS performances testing 20m
      Speaker: Doug Benjamin (Duke University (US))
    • 14:30 → 16:05
      Site Reports
      • 14:30
        BNL 5m
        Speaker: Eric Christian Lancon (CEA/IRFU,Centre d'etude de Saclay Gif-sur-Yvette (FR))
      • 14:35
        AGLT2 5m
        Speakers: Robert Ball (University of Michigan (US)), Dr Shawn McKee (University of Michigan ATLAS Group)

        We received notice from Rod Walker that we had a LOT of stageout failures.  Bob had noticed this, but had not yet gotten to an investigation.  Shawn and Bob dug in and, with some LSM consultation from Dave Lesny, we determined that something must be amiss with dCache itself.  We consequently updated from 2.13.42 to 2.13.50, with a full dCache restart, and the problem resolved.

        One dCache shelf of 20TB was retired from an old server to use the disks as (needed) spares.  Two pool servers from the RBD funds are awaiting some software attention, one at MSU and one at UM, after which they will be moved to production dCache storage state.

        The Dirty COW vulnerability patching was completed everywhere.  At the same time we updated the OSG software on the gatekeepers to version 3.3.18, with HTCondor 8.4.9.  With OSG 3.3.19 just released, containing HTCondor-CE reporting tools for AGIS, we will plan on updating again in the very near future to this version.

        We are working hard to determine remaining funds for spendout.  MSU has already done so, and has received all their new equipment (5xR630, 3xN2048 switches).  Refer to the agenda discussion notes for more information on this.

         

      • 14:40
        MWT2 5m
        Speakers: David Lesny (Univ. Illinois at Urbana-Champaign (US)), Lincoln Bryant (University of Chicago (US))

        On-going work at Chicago to update Ceph to Jewel release.

        Working on FY17 procurement.

        Dirty Cow patch applied.

        Work on Juniper -to- UC SciDMZ distribution switch in progress.

        No operational issues to speak of.

      • 14:45
        NET2 5m
        Speaker: Prof. Saul Youssef (Boston University (US))

         

        About to do purchases.

        Missing files (probably date from DQ2 days) ticket resolved by Cedric.

        We have connectivity to ESnet and LHCONE.  Yvonne and Chuck are testing the circuit.  We'll start using LHCONE in production after the Holidays.

        Planning for the new hardware.

        The NESE project (CEPH) has started up.

      • 14:50
        SWT2-OU 5m
        Speaker: Dr Horst Severini (University of Oklahoma (US))

        - all sites are running well

          - issue with OSCER nodes draining understood and fixed

        - have spent (or at least committed) all FY16 and the $50k of FY17 funds for 14 new R430s, as well as salaries and travel, to be spent by Jan 31

          - 14 R430s delivered today, will install after the break

          - will take inventory of funds early next year and make sure that any leftover funds  (if any) will be spent

        - will attend ATLAS Site Jamboree

         

      • 14:55
        SWT2-UTA 5m
        Speaker: Patrick Mcguigan (University of Texas at Arlington (US))
      • 15:00
        WT2 5m
        Speaker: Wei Yang (SLAC National Accelerator Laboratory (US))
    • 16:05 → 16:10
      AOB 5m