Joint US ATLAS Software and Computing / Physics Support Meeting

20 Aug 2014, 08:00 → 22 Aug 2014, 21:30 US/Pacific

Lawrence Berkeley National Laboratory

Erich Ward Varnes (University of Arizona (US)), Jason Nielsen (University of California,Santa Cruz (US)), Mark Neubauer (Univ. Illinois at Urbana-Champaign (US)), Torre Wenaus (Brookhaven National Laboratory (US))

This meeting combines what in the past have been three separate meetings:

• The regular series of facilities/distributed software F2F meetings
• The annual US ATLAS SW/PS planning meeting
• The annual scrubbing of our activities by US ATLAS management

US ATLAS developers and technical experts are being invited to the meeting so that it can be a substantive technical discussion and planning session for key US activities. Towards making it substantive and technical, the topics will be limited.

The meeting will be F2F only, no Vidyo.

Meeting Etherpad http://piratepad.be/p/USATLASLBL

Meeting Rooms Map map_meeting_rooms.png

Wednesday, 20 August

Welcome

Conveners: Paolo Calafiura (Lawrence Berkeley National Lab. (US)), Dr Torre Wenaus (Brookhaven National Laboratory (US))

slides Welcome_USATLAS_LBLAug14.pdf

xAOD and ROOT analysis ecosystem

EventLoop, ‘pROOT’-like functionality, xAOD analysis environment/framework, systematics framework, etc

Conveners: Dr David Malon (Argonne National Laboratory (US)), Paolo Calafiura (Lawrence Berkeley National Lab. (US))

1 Session Introduction

Speaker: Paolo Calafiura (Lawrence Berkeley National Lab. (US))

Slides USATLAS_LBL_Aug14_xAODIntro.pdf

2 What is pROOT?

Speaker: Doug Benjamin (Duke University (US))

Slides Aug14-LBL-PROOT.pdf Aug14-LBL-PROOT.pptx

3 pROOT use cases for Run II (Discussion)

Speaker: Zachary Louis Marshall (Lawrence Berkeley National Lab. (US))

Slides Marshall_pROOT_2014.08.20.pdf

4 xAOD persistence considerations: size, versioning, schema evolution

Speaker: Peter Van Gemmeren

Slides LBNL_xAOD.pdf LBNL_xAOD.pptx

5 metadata and metadata tools for analysis

Speaker: Dr Jack Cranshaw (Argonne National Laboratory (US))

Slides USATLAS.08.2014.pdf USATLAS.08.2014.pptx

10:30 Coffee Break

6 Design issues arising from CP tools migration

Speakers: Steven Andrew Farrell (University of California-Unknown-Unknown), Steven Andrew Farrell (Lawrence Berkeley National Lab. (US))

Slides Farrell_CPToolIssues.pdf

7 Supertools for systematics and analysis steering (Discussion)

Speaker: Nils Erik Krumnack (Iowa State University (US))

Slides LBL_20Aug14.pdf

8 Data Access and Dual-use Tools (Discussion)

Speaker: Paolo Calafiura (Lawrence Berkeley National Lab. (US))

Slides Analysis_Store_-_US_ATLAS_@_LBL.pdf Analysis_Store_-_US_ATLAS_@_LBL.pptx

9 Wrap-up discussion: US priorities in analysis support, discussion subjects for afternoon splinter session

Speaker: Dr David Malon (Argonne National Laboratory (US))

Slides Malon2014August20USATLAS.pdf Malon2014August20USATLAS.pptx

Lunch

Splinter

Supporting US analysis

This meeting comes shortly after the US ATLAS annual meeting where the Tier 3 white paper will be presented and discussed, so LBNL will be a good time to discuss moving forward to a focus on implementation. Topics can encompass things like data access for Tier 3s, physics support experience with FAX, ATLAS Connect, ... selectively, to make good use of 1/2 day

Conveners: Erich Ward Varnes (University of Arizona (US)), Jason Nielsen (University of California,Santa Cruz (US))

ATLAS Connect doc ATLAS Connect requirements

10 Storage of output from jobs that run remotely

Speaker: Doug Benjamin (Duke University (US))

Slides Benjamin-0819-AnalysisIO.pdf Benjamin-0819-AnalysisIO.pptx

11 ATLAS Connect from Tier3

Speaker: Robert William Gardner Jr (University of Chicago (US))

Slides ATLAS_Connect_Usability_from_a_Tier-3_(3).pdf

12 FAX tests

Speakers: Erich Ward Varnes (University of Arizona (US)), Fred Luehring (Indiana University (US))

Slides US_SW20140820.pdf

13 Remote analysis test package

Speaker: Erich Ward Varnes (University of Arizona (US))

Slides RemoteAnalysesTestPackage.key.pdf

15:10 Break

14 ATLAS Infrastructure: LS1 Upgrade Challenges

Speaker: Dr Alexander Undrus (Brookhaven National Laboratory (US))

Slides undrus_usatlas_cands_meeting_ro_Aug2014.pdf undrus_usatlas_cands_meeting_ro_Aug2014.pptx

15 PROOF/PROOF-on-demand within new Tier3 model

Speaker: Shuwei Ye (Brookhaven National Laboratory (US))

Slides yesw-Proof-PoD-20140820.pdf yesw-Proof-PoD-20140820.pptx

16 Discussion

Dinner

http://greatchinaberkeley.com

I heard they do catering and host dinners for the Chinese Consulate in San Francisco. Enough said!

Thursday, 21 August

Event Service

Conveners: Dr Torre Wenaus (Brookhaven National Laboratory (US)), Vakho Tsulaia (Lawrence Berkeley National Lab. (US))

17 PART 1: Status survey with emphasis on outstanding issues and near/midterm plans (presentation/discussion ~50/50)

18 Intro, plans, issues and objectives for today

Speaker: Dr Torre Wenaus (Brookhaven National Laboratory (US))

Slides Wenaus-eventServiceIntro-LBNL-20140821.pdf Wenaus-eventServiceIntro-LBNL-20140821.pptx

19 PanDA/JEDI’s support for fine grained dynamic job definition, event-level bookkeeping, and the event service

Speaker: Tadashi Maeno (Brookhaven National Laboratory (US))

Slides LBNL_es_pandajedi.pdf

20 Event service in the pilot: refactoring, payload integration & interaction, input/output management, object store usage

Speakers: Paul Nilsson (Brookhaven National Laboratory (US)), Wen Guan (University of Wisconsin (US))

Slides Event_Service_in_the_Pilot.pdf

21 athenaMP payload, event I/O, event index integration

Speakers: Dr Peter Van Gemmeren (Argonne National Laboratory (US)), Vakho Tsulaia (Lawrence Berkeley National Lab. (US))

Slides EvtSvc-EvtProcessing.pdf

10:20 Break

22 PART 2: Next steps for integrating, operating and scaling the ES in the early deployment targets (mainly discussion, guided by a few slides)

23 ES issues, plan and timeline: pre-production on the grid

Speakers: Paul Nilsson (Brookhaven National Laboratory (US)), Wen Guan (University of Wisconsin (US))

Slides Event_Service_Pilot_Plans.pdf

24 ES issues, plan and timeline: operating and scaling up on the Amazon spot market

Speakers: Michael Ernst, Dr Torre Wenaus (Brookhaven National Laboratory (US))

Slides AWS_pilot.pdf

25 ES issues, plan and timeline: event service on HPCs

Status, plans, technical discussion, deployment targets, timeline for the ‘Yoda’ MPI scheme

Speaker: Vakho Tsulaia (Lawrence Berkeley National Lab. (US))

Slides Yoda.pdf

26 SC14 demo objectives and plans, in light of today’s discussion

Speaker: Dr Torre Wenaus (Brookhaven National Laboratory (US))

Lunch

High Performance Computing (HPC)

Including common approach to PanDA integration and event service integration, athena porting, software access, ...

Conveners: Dr Alexei Klimentov (Brookhaven National Laboratory (US)), Kaushik De (University of Texas at Arlington (US)), Thomas Le Compte (Argonne National Laboratory (US))

27 Introduction

Speakers: Dr Alexei Klimentov (Brookhaven National Laboratory (US)), Kaushik De (University of Texas at Arlington (US)), Thomas Le Compte (Argonne National Laboratory (US))

Slides HPC_intro_US_ATLAS_AUG_2014.pdf HPC_intro_US_ATLAS_AUG_2014.pptx

28 An overview and HPC strategy

Speaker: Thomas Le Compte (Argonne National Laboratory (US))

Slides HPC_Vision.pdf HPC_Vision.ppt

29 Activities Highlights

15:05 Tea break

30 Technical Discussion

31 wrap up and next steps

Splinter

Splinter - ES/HPC Yoda technical discussion

Develop the technical specifics and workplan for the MPI based Yoda concept for event service HPC integration. Plan a timeline consistent with demoing some form of Yoda at SC14. Same room as HPC session. Duration: until we've had enough.

Conveners: Dr Torre Wenaus (Brookhaven National Laboratory (US)), Vakho Tsulaia (Lawrence Berkeley National Lab. (US))

Friday, 22 August

US ATLAS Facilities readiness for Run 2

Processing, storage management, (federated) data access, facility operations optimization, ...

Conveners: Michael Ernst (Brookhaven National Laboratory (US)), Robert William Gardner Jr (University of Chicago (US))

32 Introduction, goals for the meeting

Speakers: Michael Ernst, Robert William Gardner Jr (University of Chicago (US))

33 Priorities, adjustments for the new analysis model, derivation framework, analysis support requirements: discussion based on Wednesday & Thursday meetings

34 Tier3 Implementation Committee recommendations: recap and summary, and facility impacts

Speaker: Mark Neubauer (Univ. Illinois at Urbana-Champaign (US))

Slides LBNL_T3IC.pdf

35 Flocking from Tier3 for analysis batch, ATLAS Connect to campus clusters, etc.

Speaker: Robert William Gardner Jr (University of Chicago (US))

Slides

• ATLAS_Connect_Overview.pdf
• ceph-aug-2014.pptx
• thin-provision-t3.pdf
• thin-provision-t3.pptx

10:30 Break

36 WAN access via FAX by Panda

Speaker: Ilija Vukotic (University of Chicago (US))

Slides FAX_and_Panda_WAN_access(2).pdf https://docs.google.com/presentation/d/1c0CEdi7Q9kgPDD0i2mDiVjN5C2lzlQdtFLyJ5hemHtQ/edit?usp=sharing

37 Autopyfactory performance and configuration

Speaker: Dr Jose Caballero Bejar (Brookhaven National Laboratory (US))

Slides talk-apf-lbnl.pdf

38 HTCondor-CE production readiness

Speaker: Mr Xin Zhao (Brookhaven National Laboratory (US))

Slides HTCondor-CE-LBL.pdf

39 Dynamic multicore provisioning issues

Speaker: William Strecker-Kellogg (BNL)

Slides dynamic-slots-8-2014.pdf

40 Managing data in the facility (Group, User, LocalGroup, etc); deletions, subscriptions, etc., with focus on user capabilities

41 Prioritizing access to CPU and storage resources for US physicists

Lunch

Splinter

US ATLAS S&C/PS Management Scrubbing

This will be a closed session attended by S&C/PS and US ATLAS operations program management to examine the S&C/PS program, plan and budget, towards finalizing the FY15 program and budget.

Conveners: Jim Cochran (Iowa State Univ.), Srini Rajagopalan (Brookhaven National Laboratory (US))