US ATLAS Computing Facilities Face-to-Face at SLAC

30 Nov 2022, 13:00 → 2 Dec 2022, 13:00 US/Pacific

Build 48, Redwood Room A/B/C/D (SLAC)

Robert William Gardner Jr (University of Chicago (US)), Shawn Mc Kee (University of Michigan (US))

We are restarting our face to face meetings for US ATLAS Computing Facilities and Distributed Computing (WBS 2.3).  This will be a great opportunity to renew our collaboration, address current challenges, find new opportunities to share expertise - all while supporting the LHC Run3 physics program for ATLAS physicists while exploring facility R&D activities for HL-LHC.  

And online:

 

• Live Notes
• Facility R&D Notes

Wednesday 30 November

13:00 → 15:00 Introduction Plenary

This session will provide and introduction to the meeting and have site and activity reports to inform everyone of our current status.

Conveners: Robert William Gardner Jr (University of Chicago (US)), Shawn Mc Kee (University of Michigan (US))

welcome.pdf

welcome.pptx

13:00 Introduction and Goals of the Workshop

Speakers: Robert William Gardner Jr (University of Chicago (US)), Shawn McKee

Slides

US ATLAS Facilities SLAC F2F Meeting Introduction.pdf

13:20 Site Report: Tier1

Speakers: Doug Benjamin (Brookhaven National Laboratory (US)), Eric Christian Lancon (Brookhaven National Laboratory (US)), Vincent Garonne (Brookhaven National Laboratory (US))

2022-30-11_USATLAS T1 Storage Report_ Status & Plan.pdf

Tier1-Site-Report.pdf

14:00 Site Report: AGLT2

Wenjing will cover AGLT2, providing a quick overview of the site and then describing the unique work, tools and plans associated with the site.

Speaker: Ms Wenjing Dronen

AGLT2 F2F Site Report

AGLT2 Site Report - SLAC F2F.pdf

14:20 Site Report: MWT2

Speaker: Judith Lorraine Stephen (University of Chicago (US))

MWT2 Site Report .pdf

14:40 Site Report: NET2

Speakers: Rafael Coelho Lopes De Sa (University of Massachusetts (US)), William Axel Leight (University of Massachusetts Amherst)

NET2 Site Update.pdf

15:00 → 15:30 Break

15:30 → 17:25 Introduction Plenary

This session will provide and introduction to the meeting and have site and activity reports to inform everyone of our current status.

welcome.pdf

welcome.pptx

15:30 Site Report: SWT2

Speakers: Mark Sosebee (University of Texas at Arlington (US)), Patrick McGuigan (UTA), Patrick Mcguigan (University of Texas at Arlington (US))

SWT2-slacF2F-1.pdf

15:50 HPC Operations

Speakers: Lincoln Bryant (University of Chicago), Rui Wang (Argonne National Laboratory (US))

HPC Operations @ WBS 2.3 f2f 2022

16:05 Combined AF Session

Speakers: Fengping Hu (University of Chicago (US)), Ofer Rind (Brookhaven National Laboratory), Wei Yang (SLAC National Accelerator Laboratory (US))

2022.11.30 BNL, SLAC and UChicago Shared Analysis Facilities FtoF at SLAC.pdf

AF for Run 3

Future Challenges for the US AFs.pdf

17:05 Improving Operations in the US Cloud

Speakers: Armen Vartapetian (University of Texas at Arlington (US)), Ofer Rind (Brookhaven National Laboratory)

221130_ADCoS.pdf

Operations in the US Cloud.pdf

Thursday 1 December

08:00 → 08:30 Broad Themes and Challenges for ATLAS S&C

Speakers: Alessandro Di Girolamo (CERN), Zach Marshall (Lawrence Berkeley National Lab. (US))

20221201 ATLAS S&C in the HL-LHC world.pdf

08:30 → 10:00 Lightning Talks: Lightning Talks 1

We will have various lightning talks allowing everyone to describe their work and plans.

Conveners: Ofer Rind (Brookhaven National Laboratory), Wei Yang (SLAC National Accelerator Laboratory (US))

08:30 Data Placement Activity in ADC

Hiro will cover Data Placement Activity in ADC in terms of site storage usage efficiency and implications for our facilities.

Speaker: Hironori Ito (Brookhaven National Laboratory (US))

Data Placement Activity in ADC.pdf

08:45 Virtual Placement: A Cost/Benefit Analysis

Speaker: Ilija Vukotic (University of Chicago (US))

Virtual Placement, cost/benefit estimate

09:00 Data Challenge Testing

Hiro will describe the tools and plans to test USATLAS sites in preparation for DC24.

Speaker: Hironori Ito (Brookhaven National Laboratory (US))

Data Challenge Testing for USATLAS sites 2022 Nov .pdf

09:15 Run3 Analysis User Support in US ATLAS

Presentation about ongoing and planned user support for Run3

Speakers: Amber Roepe-Gier (University of California,Santa Cruz (US)), Mike Hance (UC Santa Cruz), Nils Erik Krumnack (Iowa State University (US))

Krumnack-2022-12-01.pdf

09:30 Analysis Facility Metrics

A lightning talk summarizing analysis facility metrics (existing and proposed).

Speakers: Ilija Vukotic (University of Chicago (US)), Mike Hance (UC Santa Cruz), Nils Erik Krumnack (Iowa State University (US))

20221201_AFMetrics.pdf

09:45 How does ServiceX work?

Speaker: Ilija Vukotic (University of Chicago (US))

ServiceX - How it works?

10:00 → 10:30 Break

10:30 → 12:00 Lightning Talks: Lightning Talks 2

We will have various lightning talks allowing everyone to describe their work and plans.

Conveners: Ofer Rind (Brookhaven National Laboratory), Wei Yang (SLAC National Accelerator Laboratory (US))

10:30 ADC Analytics Platform

Speaker: David Jordan (University of Chicago (US))

Analytics Platform at MWT2.pdf

10:45 ATLAS Canada: Infrastructure and Analytics Summary

Lightning talk on ATLAS Canada's infrastructure and use of analytics.

Speaker: Fernando Fernandez Galindo (TRIUMF (CA))

TRIUMF_AnlyticsSummary_Dec2022.pdf

TRIUMF_AnlyticsSummary_Dec2022.pdf

TRIUMF_AnlyticsSummary_Dec2022.pptx

11:00 ATLAS Canada: Overview of Cloud Native Infrastructure at U Victoria

A lightning talk on the Univ of Victoria's Cloud Native Infrastructure.

Speaker: Ryan Taylor (University of Victoria (CA))

20221201 US ATLAS Facilities F2F - UVic T2 k8s .pdf

UVic k8s-native T2

11:15 Update on the Kubernetes Project at SWT2

A lightning talk on the SWT2 investigation of running a Tier-2 computing center on the Kubernetes platform.

Speaker: Armen Vartapetian (University of Texas at Arlington (US))

221201_Kubernetes_UTA.pdf

11:30 JBOD management tools

Speaker: Robert Hancock

JBOD Management ATLAS SLAC Dec 2022 (1).PDF

JBOD Management ATLAS SLAC Dec 2022 (1).pptx

JBOD Management ATLAS SLAC Dec 2022.pptx

11:45 Deployment Experiences with EL9 at MWT2

Speaker: Farnaz Golnaraghi (University of Chicago (US))

EL9-Deployment-MWT2.pdf

12:00 → 13:00 Lunch

13:00 → 15:00 Breakouts: Breakout Sessions

Breakout sessions (in parallell)

Conveners: Robert William Gardner Jr (University of Chicago (US)), Shawn Mc Kee (University of Michigan (US))

13:00 Container Image Development

Discuss efforts to provide a coherent plan for container image management (composition, curation, lifecycle management), in particular for the Analysis Facility use cases.

This breakout will use the alternate room and the following Zoom coordinates:

https://umich.zoom.us/j/94766135812?pwd=eXVJcTlXTmpCUVBmNnF5MUhmRkF5Zz09

Speakers: Alexander Undrus (Brookhaven National Laboratory (US)), Doug Benjamin (Brookhaven National Laboratory (US)), Ilija Vukotic (University of Chicago (US)), Shuwei Ye (Brookhaven National Laboratory (US))

Breakout - Container .pdf

Link to Google slides

zoom

13:00 NET2.1 Status and Planning

The new operations and leadership team for NET2 could use advice and discussion about best practices, technology and tool options, and other items important to running and managing a Tier-2.

This breakout will use the same room and Zoom coordinates as the rest of the meeting.

Speakers: Eduardo Bach (University of Massachusetts (US)), Fred Luehring (Indiana University (US)), Rafael Antonio Lopez, Verena Ingrid Martinez Outschoorn (University of Massachusetts (US)), William Axel Leight (University of Massachusetts Amherst)

NET 2.1 Detailed Discussion.pdf

14:00 CIOPS / Federated Operations

We need to discuss how we are doing federations operations, what tools we are (or should be) using and what issues and concerns are present. A goal should be to determine if changes are required, and, if so, what effort and prototyping might be required.

This breakout will use the alternate room and the following Zoom coordinates:

https://umich.zoom.us/j/94766135812?pwd=eXVJcTlXTmpCUVBmNnF5MUhmRkF5Zz09

Outline for Discussion
https://docs.google.com/document/d/1c9bF7qIsVylHDQCu0aGyNLT9GmZvw4k81NjVjAtAUfg/edit?usp=sharing

Speakers: Ilija Vukotic (University of Chicago (US)), Ofer Rind (Brookhaven National Laboratory)

SLATE update

14:00 K8s (Kubernetes) Distributions, Deployments and Monitoring

Focused discussion K8s cluster administration and best practices for service deployment and operation.

This breakout will use the same room and Zoom coordinates as the main meeting.

Speakers: Christopher Henry Hollowell (Brookhaven National Laboratory (US)), Fengping Hu (University of Chicago (US))

bnl_okd.pdf

k8s_discussion.pdf

K8s distribution, Deployment.pdf

15:00 → 15:30 Break

Break for discussion, coffee/tea? and to allow breakout conveners to summarize their sessions.

15:30 → 16:45 Breakout Plenary

Discuss / summarize the breakout sessions, highlighting things for the whole group.

15:30 Breakout Session NET2.1 Summary and Discussion

We will have the breakout conveners present a quick summary and introduction topics for discussion.

Speakers: Fred Luehring (Indiana University (US)), Rafael Antonio Lopez, Verena Ingrid Martinez Outschoorn (University of Massachusetts (US))

NET2.1 Breakout Summary

15:50 Breakout Session on Containerization Summary and Discussion

We will have the breakout conveners present a quick summary and introduction topics for discussion.

Speakers: Alexander Undrus (Brookhaven National Laboratory (US)), Doug Benjamin (Brookhaven National Laboratory (US)), Ilija Vukotic (University of Chicago (US)), Shuwei Ye (Brookhaven National Laboratory (US))

Containerization Breakout Summary

16:05 Breakout Session on K8s Summary and Discussion

We will have the breakout conveners present a quick summary and introduction topics for discussion.

Speakers: Christopher Henry Hollowell (Brookhaven National Laboratory (US)), Fengping Hu (University of Chicago (US))

K8s Breakout Summary

16:20 Breakout Session on CIOPs/Federation Summary and Discussion

We will have the breakout conveners present a quick summary and introduction topics for discussion.

Speakers: Ilija Vukotic (University of Chicago (US)), Ofer Rind (Brookhaven National Laboratory)

FedOps Breakout Summary

16:45 → 17:15 Lightning Talks

We will have various lightning talks allowing everyone to describe their work and plans.

16:45 Storage Cost and Workflows

Lightning talk about storage planning based on the BNL presentation at the IRIS-HEP Coordinated Ecosystems meeting

Speaker: Eric Christian Lancon (Brookhaven National Laboratory (US))

Lancon-WBS2.3-Nov-2022.pdf

17:00 Optimizing Storage Utilization in the HL-LHC Era

Update on the BNL LDRD project to design an AI/ML-based, optimized data management system.

Speaker: Qiulan Huang (Brookhaven National Laboratory (US))

LDRD for ATLAS.pptx.pdf

18:30 → 20:30 Workshop Dinner

Fish Market

https://www.tripadvisor.com/Restaurant_Review-g32849-d497849-Reviews-The_Fish_Market_Palo_Alto-Palo_Alto_California.html

Friday 2 December

08:30 → 10:00 Workshop Summary and Discussions: Planning Discussions

We will present on near term plans, results from this workshop and assign follow-up homework, as appropriate.

Conveners: Robert William Gardner Jr (University of Chicago (US)), Shawn Mc Kee (University of Michigan (US))

08:30 Tier-2 Procurement Process / Milestones

Fred will discuss the proposed milestones and procurement process for our Tier-2, leaving time to discuss the main points amongst the facility.

Speaker: Fred Luehring (Indiana University (US))

Tier 2 Procurement and Milestones.pdf

09:00 Facility Data Challenge Planning

Shawn will summarize what is known about the upcoming WLCG Network Data Challenge and describe possible areas the USATLAS facilities could participate, leaving time for discussion.

Speaker: Shawn Mc Kee (University of Michigan (US))

Facility Data Challenge Planning.pdf

09:30 Extended Discussion Session

Use this slot to continue discussions about Tier-2 procurement and/or Data Challenge activities from the Facility.

We could also discuss the timeline and steps for allowing our sites to find the next operating system after RHEL7/CentOS7 becomes deprecated in 2024. Need about 10-15 minutes describing what is known and what critical areas need to be explored. Leave 15-20 minutes for discussion.

NOTE: We are gathering action items in this Google doc: https://docs.google.com/document/d/1H0hKN3M37RzVJ3DLZK437SIXdLczGCmpUWyc10KTo8Y/edit?usp=sharing

10:00 → 10:30 Break

10:30 → 11:30 Workshop Summary and Discussions: Workshop Discussion and Plans 2

We will present on near term plans, results from this workshop and assign follow-up homework, as appropriate.

Conveners: Robert William Gardner Jr (University of Chicago (US)), Shawn Mc Kee (University of Michigan (US))

10:30 Tokens, HTCondor, Storage and our Facilities

We need to lay out the issues with the evolution of our infrastructure to tokens, replacing x.509. HTCondor 10.x will remove support for x.509 and our sites should already be capable of transitioning to this. However, storage still requires x.509 and the issues around replacing x.509 with tokens should be discussed.

Brian and Ofer should summarize the issues and lead a discussion on these topics.

Speakers: Brian Hua Lin (University of Wisconsin - Madison), Ofer Rind (Brookhaven National Laboratory)

2022-12-02.osg-lhc-atlas-f2f.pdf

2022-12-02.osg-lhc-atlas-f2f.pptx

11:00 Workshop Review and Final Discussions

We need to summarize the workshop and allow follow-up discussions on any remaining topics we weren't able to spend sufficient time on.

NOTE: Let's use this Google doc to capture facility action items: https://docs.google.com/document/d/1H0hKN3M37RzVJ3DLZK437SIXdLczGCmpUWyc10KTo8Y/edit?usp=sharing

11:30 → 12:00 Workshop Closeout: Closeout and Homework Assignments

11:30 Homework and action items

As a group, define the actions items identified in the meeting, assigning names and timelines when appropriate

11:50 Closeout

Speakers: Robert William Gardner Jr (University of Chicago (US)), Shawn Mc Kee (University of Michigan (US))

slides