US ATLAS Software and Computing Technical Planning Meeting 2015

28 Jun 2015, 10:00 → 29 Jun 2015, 13:00 Europe/Zurich

40/S2-D01 - Salle Dirac (CERN)

Mark Neubauer (Univ. Illinois at Urbana-Champaign (US)), Michael Ernst (Unknown), Nurcan Ozturk (University of Texas at Arlington (US)), Torre Wenaus (Brookhaven National Laboratory (US))

This meeting combines

• one in the regular series of facilities/distributed software F2F meetings
• the annual US ATLAS software technical planning meeting

Developers and technical experts are invited to this meeting so that it can be a substantive technical discussion and planning session for key US activities. Towards making it substantive and technical, the topics will be limited.

All ATLAS members interested to attend are welcome.

A follow-up management meeting at Duke in July will factor the outcomes of this meeting into the planning for the next year, and will scrub the S&C and Physics Support programs.

Sorry about the pin, it is 0628

Meeting notes - editable googledoc -- please contribute

Sunday 28 June

10:00 → 13:15 Core services, event store

• MT framework and the US role, common efforts (CMS?), US role in Gaudi(Hive), G4Hive
• where next for athenaMP
• breaking G4 simu away from athena, statically linked G4 with performance improvement, G4MT
• athena platform porting (powerPC...)
• where next for event I/O, in-file metadata, US role
• where next for ROOT6
• event streaming service: specific objectives, workplan and timeline
• US planning issues for the rel 21 schedule
• longer term program to run 4

Conveners: David Malon (Argonne National Laboratory (US)), Nurcan Ozturk (University of Texas at Arlington (US)), Paolo Calafiura (Lawrence Berkeley National Lab. (US))

10:00 Future Framework

Speaker: Charles Leggett (Lawrence Berkeley National Lab. (US))

Slides FutureFrameworks.pdf

10:40 Multiprocessing in Run 2 and beyond

Speaker: Vakho Tsulaia (Lawrence Berkeley National Lab. (US))

Slides AthenaMP.pdf

11:00 Machine Learning and Run 4 Computing

Speaker: Paolo Calafiura (Lawrence Berkeley National Lab. (US))

Slides USATLAS_WorkshopJun15_ML.pdf USATLAS_WorkshopJun15_ML.pptx

11:15 Coffee Break

11:30 Toward a next-generation I/O framework

integrated into ATLAS future framework, removing the last vestiges of POOL/APR, supporting mulithreading and distributed processing, suitable for HPC deployment

Speakers: Alexandre Vaniachine (ANL), David Malon (Argonne National Laboratory (US)), Jack Cranshaw (Argonne National Laboratory (US)), Marcin Nowak (Brookhaven National Laboratory (US)), Peter Van Gemmeren (Argonne National Laboratory (US))

A framework-oriented bridge to metadata discussions Malon2015June28Metadata.pdf Malon2015June28Metadata.pptx

Toward a next-generation I/O framework Malon2015June28IO.pdf Toward a next-generation I/O framework

12:15 The future of metadata and bookkeeping infrastructure

why it matters, what the issues are, near-term and longer-term work

Speakers: Alexandre Vaniachine (ANL), David Malon (Argonne National Laboratory (US)), Jack Cranshaw (Argonne National Laboratory (US)), Marcin Nowak (Brookhaven National Laboratory (US)), Peter Van Gemmeren (Argonne National Laboratory (US))

File Peeking FilePeeking.pdf

Post Reco US_Jun2015_Pre_Meta_2.pdf US_Jun2015_Pre_Meta_2.pptx

14:00 → 16:15 Distributed software, HPC

• where next for PanDA/JEDI, Prodsys2, monitoring, UberPanDA
• collaborative work on UberPanDA (=catch-all for Big, Mega, Exa etc), benefits to/relation with ATLAS work
• plans for new developments, psub, distributed aspects of event streaming service, data product catalog, ...
• US HPCs, ops program roles, relation to non-ops & other ATLAS work, convergence. Objectives, timeline for US HPC prodsys production

Conveners: Alexei Klimentov (Brookhaven National Laboratory (US)), Kaushik De (University of Texas at Arlington (US)), Nurcan Ozturk (University of Texas at Arlington (US))

14:00 Introduction and outline of the session

Speakers: Alexei Klimentov (Brookhaven National Laboratory (US)), Kaushik De (University of Texas at Arlington (US))

Slides US_ATLAS_Meeting_Jun28_2015_DistSW-kd.pdf US_ATLAS_Meeting_Jun28_2015_DistSW-kd.pptx

14:10 Distributed Software

Speakers: Alden Stradling (University of Texas at Arlington (US)), Alexandre Vaniachine (ATLAS), Paul Nilsson (Brookhaven National Laboratory (US)), Sasha Vanyashin (Argonne National Laboratory (US)), Siarhei Padolski (BNL), Tadashi Maeno (Brookhaven National Laboratory (US))

15:05 Distributed SW Q&A. Discussion

15:15 HPC&LCF

Speakers: Danila Oleynik (Joint Inst. for Nuclear Research (RU)), Sergey Panitkin (Brookhaven National Laboratory (US)), Taylor Childers (Argonne National Laboratory (US)), Vakho Tsulaia (Lawrence Berkeley National Lab. (US))

15:55 HPC&LCF Q&A

16:00 → 16:30 Software Infrastructure

Convener: Alexander Undrus (Brookhaven National Laboratory (US))

16:00 ATLAS software infrastructure: requirements and goals at Run 2 period

Speaker: Alexander Undrus (Brookhaven National Laboratory (US))

Slides undrus_usatlas_cands_meeting_June2015.pdf undrus_usatlas_cands_meeting_June2015.pptx

16:15 Challenges of ATLAS software environment setup

Speaker: Shuwei Ye (Brookhaven National Laboratory (US))

Slides yesw-AnaSetup-20150628.pdf

16:30 → 18:30 Tier 3 Round 1

• the needs, the plan, the technologies and approaches, the ops program role
• LOCALGROUPDISK policy in support of US analysis
• talk things through to the point where we're able to write down an ops program plan

Conveners: Doug Benjamin (Duke University (US)), Erich Ward Varnes (University of Arizona (US)), Michael Ernst (Brookhaven National Laboratory (US))

Monday 29 June

08:25 → 11:30 US ATLAS facilities, distributed computing

• integrated facility issues & plans for supporting Run 2+ computing
• what's next for FAX, xrootd, federated data access, wide area data access
  • a cross-cutting topic that needs to involve core sw, dist sw also
• what's next for APF
• plans & expectations for opportunistic platforms: Amazon spot market, OSG, ...

Conveners: Michael Ernst (Brookhaven National Laboratory (US)), Robert William Gardner Jr (University of Chicago (US))

08:25 Introduction and Session Outline

Speaker: Michael Ernst (Brookhaven National Laboratory (US))

08:30 What's next for FAX, brokering, federated wide area data access?

Speakers: Ilija Vukotic (University of Chicago (US)), Kaushik De (University of Texas at Arlington (US))

Slides Brokering__for_WAN_access.pdf https://docs.google.com/presentation/d/1H5738Qt-IaTFYgah3EoTgEvsor1i8JjZYgMovslvZrc/edit#slide=id.p

09:00 Client distributed IO performance discussion: framework, ROOT, xrootd, ...

Speakers: Andrew Hanushevsky (STANFORD LINEAR ACCELERATOR CENTER), David Malon (Argonne National Laboratory (US)), Ilija Vukotic (University of Chicago (US))

Slides Client_distributed_IO_performance_discussion-_framework,_ROOT,_xrootd,_....pdf https://docs.google.com/presentation/d/1WjMCIPNOEJFxvxHDw23FycYHZzgw7stAxx0dnFYVtDY/edit?usp=sharing

09:20 APF News and Plans & Expectations for Opportunistic Platforms

Speakers: John Raymond Hover (Brookhaven National Laboratory (US)), Jose Caballero Bejar (Brookhaven National Laboratory (US))

Slides RACF_USATLAS_Workshop_June_29_2015.pdf

09:40 Plans & expectations for opportunistic platforms: OSG, ATLAS Connect

Speaker: Robert William Gardner Jr (University of Chicago (US))

Slides OSG_Opportunistic_ATLAS_Connect_(1).pdf

10:00 Xrootd Caching

Speakers: Ilija Vukotic (University of Chicago (US)), Wei Yang (SLAC National Accelerator Laboratory (US))

Slides https://docs.google.com/presentation/d/1JiM1H8iJPKTJfWvtqotiD1IDL0jI5hmYCIM03UMUjdo/edit#slide=id.p Xrootd_Caching.pdf

10:20 SDN Provisioning, next steps after ANSE

Speaker: Kaushik De (University of Texas at Arlington (US))

Slides US-SDN-June15.pdf US-SDN-June15.ppt

11:30 → 12:30 Tier 3 round 2

• write down and discuss a specific ops program plan

Conveners: Doug Benjamin (Duke University (US)), Erich Ward Varnes (University of Arizona (US)), Michael Ernst (Brookhaven National Laboratory (US))