Joint US ATLAS Software and Computing / Physics Support Meeting

20 Aug 2014, 08:00 → 22 Aug 2014, 21:30 US/Pacific

Lawrence Berkeley National Laboratory

Erich Ward Varnes (University of Arizona (US)), Jason Nielsen (University of California,Santa Cruz (US)), Mark Neubauer (Univ. Illinois at Urbana-Champaign (US)), Torre Wenaus (Brookhaven National Laboratory (US))

This meeting combines what in the past have been three separate meetings:

• The regular series of facilities/distributed software F2F meetings
• The annual US ATLAS SW/PS planning meeting
• The annual scrubbing of our activities by US ATLAS management

US ATLAS developers and technical experts are being invited to the meeting so that it can be a substantive technical discussion and planning session for key US activities. Towards making it substantive and technical, the topics will be limited.

The meeting will be F2F only, no Vidyo.

Meeting Etherpad http://piratepad.be/p/USATLASLBL

Meeting Rooms Map map_meeting_rooms.png

Wednesday 20 August

08:50 → 09:00 Welcome

Conveners: Paolo Calafiura (Lawrence Berkeley National Lab. (US)), Dr Torre Wenaus (Brookhaven National Laboratory (US))

slides Welcome_USATLAS_LBLAug14.pdf

09:00 → 21:00 xAOD and ROOT analysis ecosystem

EventLoop, ‘pROOT’-like functionality, xAOD analysis environment/framework, systematics framework, etc

Conveners: Dr David Malon (Argonne National Laboratory (US)), Paolo Calafiura (Lawrence Berkeley National Lab. (US))

09:00 Session Introduction

Speaker: Paolo Calafiura (Lawrence Berkeley National Lab. (US))

Slides USATLAS_LBL_Aug14_xAODIntro.pdf

09:10 What is pROOT?

Speaker: Doug Benjamin (Duke University (US))

Slides Aug14-LBL-PROOT.pdf Aug14-LBL-PROOT.pptx

09:30 pROOT use cases for Run II (Discussion)

Speaker: Zachary Louis Marshall (Lawrence Berkeley National Lab. (US))

Slides Marshall_pROOT_2014.08.20.pdf

09:50 xAOD persistence considerations: size, versioning, schema evolution

Speaker: Peter Van Gemmeren

Slides LBNL_xAOD.pdf LBNL_xAOD.pptx

10:10 metadata and metadata tools for analysis

Speaker: Dr Jack Cranshaw (Argonne National Laboratory (US))

Slides USATLAS.08.2014.pdf USATLAS.08.2014.pptx

10:30 Coffee Break

10:50 Design issues arising from CP tools migration

Speakers: Steven Andrew Farrell (University of California-Unknown-Unknown), Steven Andrew Farrell (Lawrence Berkeley National Lab. (US))

Slides Farrell_CPToolIssues.pdf

11:10 Supertools for systematics and analysis steering (Discussion)

Speaker: Nils Erik Krumnack (Iowa State University (US))

Slides LBL_20Aug14.pdf

11:40 Data Access and Dual-use Tools (Discussion)

Speaker: Paolo Calafiura (Lawrence Berkeley National Lab. (US))

Slides Analysis_Store_-_US_ATLAS_@_LBL.pdf Analysis_Store_-_US_ATLAS_@_LBL.pptx

12:00 Wrap-up discussion: US priorities in analysis support, discussion subjects for afternoon splinter session

Speaker: Dr David Malon (Argonne National Laboratory (US))

Slides Malon2014August20USATLAS.pdf Malon2014August20USATLAS.pptx

12:30 → 13:30 Lunch

13:30 → 17:30 Splinter

13:30 → 17:30 Supporting US analysis

This meeting comes shortly after the US ATLAS annual meeting where the Tier 3 white paper will be presented and discussed, so LBNL will be a good time to discuss moving forward to a focus on implementation. Topics can encompass things like data access for Tier 3s, physics support experience with FAX, ATLAS Connect, ... selectively, to make good use of 1/2 day

Conveners: Erich Ward Varnes (University of Arizona (US)), Jason Nielsen (University of California,Santa Cruz (US))

ATLAS Connect doc ATLAS Connect requirements

13:30 Storage of output from jobs that run remotely

Speaker: Doug Benjamin (Duke University (US))

Slides Benjamin-0819-AnalysisIO.pdf Benjamin-0819-AnalysisIO.pptx

14:00 ATLAS Connect from Tier3

Speaker: Robert William Gardner Jr (University of Chicago (US))

Slides ATLAS_Connect_Usability_from_a_Tier-3_(3).pdf

14:30 FAX tests

Speakers: Erich Ward Varnes (University of Arizona (US)), Fred Luehring (Indiana University (US))

Slides US_SW20140820.pdf

14:50 Remote analysis test package

Speaker: Erich Ward Varnes (University of Arizona (US))

Slides RemoteAnalysesTestPackage.key.pdf

15:10 Break

15:40 ATLAS Infrastructure: LS1 Upgrade Challenges

Speaker: Dr Alexander Undrus (Brookhaven National Laboratory (US))

Slides undrus_usatlas_cands_meeting_ro_Aug2014.pdf undrus_usatlas_cands_meeting_ro_Aug2014.pptx

16:10 PROOF/PROOF-on-demand within new Tier3 model

Speaker: Shuwei Ye (Brookhaven National Laboratory (US))

Slides yesw-Proof-PoD-20140820.pdf yesw-Proof-PoD-20140820.pptx

16:30 Discussion

19:00 → 21:00 Dinner

http://greatchinaberkeley.com

I heard they do catering and host dinners for the Chinese Consulate in San Francisco. Enough said!

Thursday 21 August

08:59 → 12:30 Event Service

Conveners: Dr Torre Wenaus (Brookhaven National Laboratory (US)), Vakho Tsulaia (Lawrence Berkeley National Lab. (US))

09:00 PART 1: Status survey with emphasis on outstanding issues and near/midterm plans (presentation/discussion ~50/50)

09:01 Intro, plans, issues and objectives for today

Speaker: Dr Torre Wenaus (Brookhaven National Laboratory (US))

Slides Wenaus-eventServiceIntro-LBNL-20140821.pdf Wenaus-eventServiceIntro-LBNL-20140821.pptx

09:15 PanDA/JEDI’s support for fine grained dynamic job definition, event-level bookkeeping, and the event service

Speaker: Tadashi Maeno (Brookhaven National Laboratory (US))

Slides LBNL_es_pandajedi.pdf

09:30 Event service in the pilot: refactoring, payload integration & interaction, input/output management, object store usage

Speakers: Paul Nilsson (Brookhaven National Laboratory (US)), Wen Guan (University of Wisconsin (US))

Slides Event_Service_in_the_Pilot.pdf

09:55 athenaMP payload, event I/O, event index integration

Speakers: Dr Peter Van Gemmeren (Argonne National Laboratory (US)), Vakho Tsulaia (Lawrence Berkeley National Lab. (US))

Slides EvtSvc-EvtProcessing.pdf

10:20 Break

10:35 PART 2: Next steps for integrating, operating and scaling the ES in the early deployment targets (mainly discussion, guided by a few slides)

10:36 ES issues, plan and timeline: pre-production on the grid

Speakers: Paul Nilsson (Brookhaven National Laboratory (US)), Wen Guan (University of Wisconsin (US))

Slides Event_Service_Pilot_Plans.pdf

10:50 ES issues, plan and timeline: operating and scaling up on the Amazon spot market

Speakers: Michael Ernst, Dr Torre Wenaus (Brookhaven National Laboratory (US))

Slides AWS_pilot.pdf

11:20 ES issues, plan and timeline: event service on HPCs

Status, plans, technical discussion, deployment targets, timeline for the ‘Yoda’ MPI scheme

Speaker: Vakho Tsulaia (Lawrence Berkeley National Lab. (US))

Slides Yoda.pdf

12:20 SC14 demo objectives and plans, in light of today’s discussion

Speaker: Dr Torre Wenaus (Brookhaven National Laboratory (US))

12:30 → 13:30 Lunch

13:25 → 17:10 High Performance Computing (HPC)

Including common approach to PanDA integration and event service integration, athena porting, software access, ...

Conveners: Dr Alexei Klimentov (Brookhaven National Laboratory (US)), Kaushik De (University of Texas at Arlington (US)), Thomas Le Compte (Argonne National Laboratory (US))

13:30 Introduction

Speakers: Dr Alexei Klimentov (Brookhaven National Laboratory (US)), Kaushik De (University of Texas at Arlington (US)), Thomas Le Compte (Argonne National Laboratory (US))

Slides HPC_intro_US_ATLAS_AUG_2014.pdf HPC_intro_US_ATLAS_AUG_2014.pptx

13:35 An overview and HPC strategy

Speaker: Thomas Le Compte (Argonne National Laboratory (US))

Slides HPC_Vision.pdf HPC_Vision.ppt

14:05 Activities Highlights

15:05 Tea break

15:20 Technical Discussion

16:35 wrap up and next steps

13:30 → 17:30 Splinter

17:30 → 19:30 Splinter - ES/HPC Yoda technical discussion

Develop the technical specifics and workplan for the MPI based Yoda concept for event service HPC integration. Plan a timeline consistent with demoing some form of Yoda at SC14. Same room as HPC session. Duration: until we've had enough.

Conveners: Dr Torre Wenaus (Brookhaven National Laboratory (US)), Vakho Tsulaia (Lawrence Berkeley National Lab. (US))

Friday 22 August

09:00 → 12:30 US ATLAS Facilities readiness for Run 2

Processing, storage management, (federated) data access, facility operations optimization, ...

Conveners: Michael Ernst (Brookhaven National Laboratory (US)), Robert William Gardner Jr (University of Chicago (US))

09:00 Introduction, goals for the meeting

Speakers: Michael Ernst, Robert William Gardner Jr (University of Chicago (US))

09:10 Priorities, adjustments for the new analysis model, derivation framework, analysis support requirements: discussion based on Wednesday & Thursday meetings

09:50 Tier3 Implementation Committee recommendations: recap and summary, and facility impacts

Speaker: Mark Neubauer (Univ. Illinois at Urbana-Champaign (US))

Slides LBNL_T3IC.pdf

10:10 Flocking from Tier3 for analysis batch, ATLAS Connect to campus clusters, etc.

Speaker: Robert William Gardner Jr (University of Chicago (US))

Slides

• ATLAS_Connect_Overview.pdf
• ceph-aug-2014.pptx
• thin-provision-t3.pdf
• thin-provision-t3.pptx

10:30 Break

10:50 WAN access via FAX by Panda

Speaker: Ilija Vukotic (University of Chicago (US))

Slides FAX_and_Panda_WAN_access(2).pdf https://docs.google.com/presentation/d/1c0CEdi7Q9kgPDD0i2mDiVjN5C2lzlQdtFLyJ5hemHtQ/edit?usp=sharing

11:05 Autopyfactory performance and configuration

Speaker: Dr Jose Caballero Bejar (Brookhaven National Laboratory (US))

Slides talk-apf-lbnl.pdf

11:20 HTCondor-CE production readiness

Speaker: Mr Xin Zhao (Brookhaven National Laboratory (US))

Slides HTCondor-CE-LBL.pdf

11:35 Dynamic multicore provisioning issues

Speaker: William Strecker-Kellogg (BNL)

Slides dynamic-slots-8-2014.pdf

11:50 Managing data in the facility (Group, User, LocalGroup, etc); deletions, subscriptions, etc., with focus on user capabilities

12:10 Prioritizing access to CPU and storage resources for US physicists

12:30 → 13:30 Lunch

13:30 → 17:30 Splinter

13:30 → 17:30 US ATLAS S&C/PS Management Scrubbing

This will be a closed session attended by S&C/PS and US ATLAS operations program management to examine the S&C/PS program, plan and budget, towards finalizing the FY15 program and budget.

Conveners: Jim Cochran (Iowa State Univ.), Srini Rajagopalan (Brookhaven National Laboratory (US))