US ATLAS Computing Integration and Operations

Wednesday 18 Jul 2018, 13:00 → 15:00 US/Eastern

Notes and other material available in the US ATLAS Integration Program Twiki

13:00 → 13:05 Top of the Meeting

Speakers: Eric Christian Lancon (BNL), Robert William Gardner Jr (University of Chicago (US))

Top of meeting:

• Discussion of protocols at CHEP - during the data transfer session.
• SLAC meeting August 28-30.  There is a draft agenda, few presentations.  Registration is still open.  Remote participation?
  • Kaushik:
    • Tuesday, 28th - special topics (2) - physics support, hl-lhc
    • Wednesday, 29 - special topics (2) - software
    • WBS reports to level 3, presentations, HL-LHC area
    • Doug: Will we sort out WBS 2.3 and 2.4 before the meeting?  Kaushik: yes sure.
• FY18Q3 reporting
  • need to update http://bit.ly/usatlas-capacity
• SLATE for XCache deployment - presentation today

13:05 → 13:20 SLATE status

Speaker: Lincoln Bryant (University of Chicago (US))

SLATE for ATLAS.pdf

13:15 → 13:20 ADC news and issues

Speakers: Robert Ball (University of Michigan (US)), Xin Zhao (Brookhaven National Laboratory (US))

• all sites please double check/update HS06 numbers on OIM, make sure ATLAS dashboard reports your walltime hours HS06 numbers correctly
• evaluation of "underlay" feature of singularity

13:20 → 13:25 OSG software issues

Speaker: Brian Lin (University of Wisconsin)

13:25 → 13:30 Production

Speaker: Mark Sosebee (University of Texas at Arlington (US))

shift-summary-6_27_18.pdf

shift-summary-7_11_18.pdf

shift-summary-7_18_18.pdf

shift-summary-7_4_18.pdf

13:30 → 13:35 Data Management

Speaker: Armen Vartapetian (University of Texas at Arlington (US))

13:35 → 13:40 Data transfers

Speaker: Hironori Ito (Brookhaven National Laboratory (US))

13:40 → 13:45 Networks

Speaker: Dr Shawn McKee (University of Michigan ATLAS Group)

New perfSONAR version 4.1 in release candidate testing now.  Release could be within a few weeks.  Sites will need to upgrade to using CentOS 7.x to run 4.1.  If you update your OS now you can 'yum update' 4.0 instances on CentOS 7.x

New version of Mesh-config (now called PWA: pSConfig Web Administrator) installed.  Production: https://psconfig.opensciencegrid.org   ITB: https://psconfig-itb.opensciencegrid.org

 

 

13:45 → 13:50 XCache

Speakers: Andrew Bohdan Hanushevsky (SLAC National Accelerator Laboratory (US)), Andrew Hanushevsky (STANFORD LINEAR ACCELERATOR CENTER), Andrew Hanushevsky, Ilija Vukotic (University of Chicago (US)), Wei Yang (SLAC National Accelerator Laboratory (US))

13:50 → 13:55 HPCs integration

Speaker: Doug Benjamin (Duke University (US))

13:55 → 14:30 Site Reports

13:55 BNL

Speaker: Xin Zhao (Brookhaven National Laboratory (US))

• disabled singularity overlay option on all T1 WNs, because of reported security vulnerabilities, no impact to jobs
• finally HS06 numbers among OIM/AGIS/dashboard are in sync for BNL
• dCache team developed and applied procedure to detect/remove ATLAS dark data, significant improvement on dashboard w.r.t. dark data on BNL dCache

14:00 AGLT2

Speakers: Robert Ball (University of Michigan (US)), Dr Shawn McKee (University of Michigan ATLAS Group)

All operations are proceeding smoothly.

SL6 PandaQueues are now offline via OSG downtime until they can be permanently deleted.

Problem with active-active LAGs under SL7.5 was identified yesterday.  A temporary workaround is to add "miimon=100" to the BONDING_OPTS parameter of the ifcfg file.

All purchased hardware is up and running on our clusters.  The Facilities spreadsheet is up to date.

 

14:05 MWT2

Speaker: Judith Lorraine Stephen (University of Chicago (US))

Overall the site is running well and is full of jobs.

UC

• C6420 disk upgrade
  • 10x C6420s rebuilt with SSDs and PCIe adapters
  • 10x C6420s rebuilt with 4x 1TB HDDs
• Storage
  • 2x R730s and 16x MD1200s received and racked
  • Still expecting 1x R730 and 2x MD1200s (ordered later than the other equipment)
  • In the process of configuring and benchmarking
• CentOS7 upgrade: still in progress

IU

• C6420 disk upgrade
  • 8x C6420s rebuilt with SSDs and PCIe adapters
  • 8x C6420s rebuilt with 4x 1TB HDDs

UIUC

• ICC upgrade to CentOS7 scheduled for August

Singularity upgraded to 2.5.2-1 across the cluster.

14:10 NET2

Speaker: Prof. Saul Youssef (Boston University (US))

14:15 SWT2-OU

Speaker: Dr Horst Severini (University of Oklahoma (US))

- nothing to report, all sites running well

- collecting new quotes for next compute node purchases, looking for suggestions on configurations

 

14:20 SWT2-UTA

Speaker: Patrick Mcguigan (University of Texas at Arlington (US))

1) Installed the latest hardware additions at SWT2_CPB: 

• 28 compute nodes
• 400 TB storage

2) Working with campus networking staff to address an issue related to slowness/timeouts in the DNS.

14:25 WT2

Speaker: Wei Yang (SLAC National Accelerator Laboratory (US))

14:30 → 14:35 AOB