IRIS-HEP AGC workshop 2023

US/Central
UW-Madison Data Science Institute

UW-Madison Data Science Institute

McArdle Building, 11th floor 447 Lorch Street Madison, WI (United States)
Alexander Held (University of Wisconsin Madison (US)), Brian Paul Bockelman (University of Wisconsin Madison (US)), Kyle Stuart Cranmer (University of Wisconsin Madison (US)), Oksana Shadura (University of Nebraska Lincoln (US))
Description

What is this workshop?

The Analysis Grand Challenge (AGC) workshop is a three-day event, hosted from May 3–5 at the University of Wisconsin–Madison's Data Science Institute. It aims to bring together experts and all those interested to survey the current status of the AGC project and to build a concrete plan for addressing remaining items towards an AGC showcase event later this summer. Meetings will be held at the Data Science Institute, located on the 11th floor of the McArdle Building at 447 Lorch Street.

Workshop structure

The current agenda is preliminary. We will spend Wednesday and Thursday reviewing the current status of various areas connected to the AGC, with a specific focus on performance and usability. Friday morning will be dedicated to planning the next steps towards an AGC showcase event.

While some parts of this workshop will feature demonstrations similar to the AGC Tools workshops (#1, #2) or the AGC demo days (#1, #2), other parts will be more technical in nature.

The workshop will end on Friday at lunch time to allow those interested in attending the pre-CHEP workshop (starting Saturday afternoon) to fly either Friday evening or Saturday morning.

A workshop dinner will be organized on Thursday evening.

Attending the workshop in Madison

All sessions will be held on the 11th floor of the McArdle Building (447 Lorch Street), at the Data Science Institute. The number of available spots for in-person attendance of this workshop is limited by room capacity. They are distributed on a first come, first served basis, so please sign up now if you would like to join!

Please see the dedicated page on the agenda for information related to travel and accommodation. We have also prepared a convenient map with key places and food options.

Connecting remotely via Zoom

A Zoom connection is also available: you can access the link via the timetable on the left, make sure you are logged into indico to see it. Some hands-on / hackathon style parts of the agenda may not work well over Zoom, so please consider joining us in person if possible.

What is the AGC?

The AGC is a project organized by IRIS-HEP and partners. It defines physics analysis tasks at realistic scale and complexity, including the handling of systematic uncertainties, binned statistical analysis, reinterpretation, and end-to-end optimization, with a focus on techniques envisioned for use at the HL-LHC. The AGC also provides implementations addressing those tasks, making use of new and advanced analysis tools developed by the community in the Python ecosystem. It relies on the development of the required cyberinfrastructure to be executed at scale.

Videoconference
IRIS-HEP AGC demonstration 2023
Zoom Meeting ID
62599182131
Host
Alexander Held
Alternative hosts
Brian Paul Bockelman, Oksana Shadura
Useful links
Join via phone
Zoom URL
Registration
Participants
  • Alex Golub
  • Alexander Held
  • Aman Desai
  • Andrew Wightman
  • Benjamin Galewsky
  • Benjamin Tovar
  • Brian Lin
  • Brian Paul Bockelman
  • Burt Holzman
  • Carl Lundstedt
  • Carl Vuosalo
  • Christoph Paus
  • David Martin Koch
  • Doug Benjamin
  • Elise Chavez
  • Elizabeth Sexton-Kennedy
  • Elliott Kauffman
  • Enrico Guiraud
  • Fengping Hu
  • Gordon Watts
  • Ianna Osborne
  • Ilija Vukotic
  • Imanol Corredoira
  • Jaydip Singh
  • Jayjeet Chakraborty
  • John Thiltges
  • Kyle Stuart Cranmer
  • Kyungeon Choi
  • Lara Calic
  • Lincoln Bryant
  • Lindsey Gray
  • Marc Tost
  • Mariarosaria D'Alfonso
  • Mason Proffitt
  • Matthew Drnevich
  • Matthew Feickert
  • Matyas Selmeci
  • Mohamed Elashri
  • Munera Alrashed
  • Nibedita Dash
  • Nick Manganelli
  • Nick Smith
  • Ofer Rind
  • Oksana Shadura
  • Robert William Gardner Jr
  • Ryan Simeon
  • Samuel Albin
  • Sanjeev Kumar
  • Shashi Kumar Samdarshi
  • Shivani Lomte
  • Shruti De
  • Sourav Patra
  • Tal van Daalen
  • Tatiana Ovsiannikova
  • Varun Sharma
    • Morning session: Analysis Systems and demos
      Convener: Matthew Feickert (University of Wisconsin Madison (US))
    • 12:30 PM
      Lunch break
    • Afternoon session: ServiceX, AGC for ATLAS + CMS
      Conveners: Benjamin Galewsky (Univ. Illinois at Urbana Champaign (US)), Gordon Watts (University of Washington (US))
      • 7
        User experience

        including demo, func-adl + ServiceX client + interacting with servers

        Speaker: Tal van Daalen (University of Washington (US))
      • 8
        Practical workflow experience
        Speakers: Kyungeon Choi (University of Texas at Austin (US)), Marc Tost (University of Texas at Austin (US))
      • 9
        Status of technical aspects

        JWT, dashboard, error reporting

        Speaker: Benjamin Galewsky (Univ. Illinois at Urbana Champaign (US))
      • 3:30 PM
        Coffee break
      • 10
        ServiceX for ATLAS

        including PHYSLITE / PHYS demo

        Speakers: Gordon Watts (University of Washington (US)), Tal van Daalen (University of Washington (US))
      • 11
        Column joining + caching strategies discussion

        planning towards NanoAOD / MiniAOD (and generic) column joining

        setup for use in CMS version of AGC

        Speaker: Lindsey Gray (Fermi National Accelerator Lab. (US))
    • Morning session: DOMA, facilities and performance
      Convener: Brian Paul Bockelman (University of Wisconsin Madison (US))
      • 12
        DOMA R&D and connections to AGC
        Speakers: Jayjeet Chakraborty, Jayjeet Chakraborty (University of California, Santa Cruz)
      • 13
        End-to-end AGC walkthrough with facility focus

        including ServiceX demo: ServiceX + JWT at coffea-casa, writing output to shared FS, dashboard / JupyterLab plugin, ML components at coffea-casa (Triton, MLFlow)

        Speakers: Andrew Wightman, Andrew Wightman (University of Nebraska Lincoln (US))
      • 10:30 AM
        Coffee break
      • 14
        Benchmarking discussion

        local vs remote files, XCache (including https status), scaling

        Speakers: Fengping Hu (University of Chicago (US)), Ilija Vukotic (University of Chicago (US))
      • 15
        Workqueue in the context of AGC
        Speaker: Benjamin Tovar Lopez (University of Notre Dame)
    • 12:30 PM
      Lunch break
    • Afternoon session: AF reports and future directions
      Convener: Oksana Shadura (University of Nebraska Lincoln (US))
      • 16
        Experience with AGC at German facilities
        Speakers: David Koch, David Martin Koch (Ludwig Maximilians Universitat (DE))
      • 17
        AGC at US CMS AFs
        Speaker: Carl Lundstedt (University of Nebraska Lincoln (US))
      • 18
        AGC at US ATLAS AFs
        Speaker: Matthew Feickert (University of Wisconsin Madison (US))
      • 3:20 PM
        Coffee break
      • 19
        Brainstorming: directions & UX for AS

        few slides about future directions to prompt discussion

        Speakers: Alexander Held (University of Wisconsin Madison (US)), Matthew Feickert (University of Wisconsin Madison (US))
      • 20
        Future plans for ServiceX

        few slides to prompt discussion (+ performance / latency?)

        Speakers: Benjamin Galewsky (Univ. Illinois at Urbana Champaign (US)), Gordon Watts (University of Washington (US))
      • 21
        Next steps for coffea-casa AF

        Discussion about sustainability, the current deployment model and design of architecture

        Speakers: Carl Lundstedt (University of Nebraska Lincoln (US)), Oksana Shadura (University of Nebraska Lincoln (US)), Sam Albin (UNL)
    • 7:00 PM
      Workshop dinner
    • Planning session: towards an AGC showcase event
      Conveners: Alexander Held (University of Wisconsin Madison (US)), Oksana Shadura (University of Nebraska Lincoln (US))
      • 22
        AS: workshop outcomes and action items

        including concrete plan towards a showcase event

        Speaker: Matthew Feickert (University of Wisconsin Madison (US))
      • 23
        DOMA: workshop outcomes and action items

        including concrete plan towards a showcase event

        Speaker: Brian Paul Bockelman (University of Wisconsin Madison (US))
      • 24
        SSL: workshop outcomes and action items

        including concrete plan towards a showcase event

        Speaker: Robert William Gardner Jr (University of Chicago (US))
      • 10:30 AM
        Coffee break
      • 25
        Discussion: towards an AGC showcase event
        Speaker: everyone !
      • 26
        Closing
        Speakers: Alexander Held (University of Wisconsin Madison (US)), Oksana Shadura (University of Nebraska Lincoln (US))