Mini-workshop: Event processing software systems

Moussala Ballroom, 1st floor (Hilton Hotel)

Moussala Ballroom, 1st floor

Hilton Hotel

Sofia, Bulgaria (Bulevard "Bulgaria" 1, 1421 g.k. Lozenets, Sofia, Bulgaria)
Charles Leggett (Lawrence Berkeley National Lab (US)), James Kowalkowski (Fermi National Accelerator Lab. (US)), Paolo Calafiura (University of California Berkeley (US))

A fundamental element of collaborative experimental data processing infrastructure software is the event processing framework.   For computationally expensive stages of data processing, the framework has heavily influenced how data is organized, viewed, and stored, and how algorithms are organized, configured, and scheduled.  Design features have, to some extent, been dictated by computing models and available computing facilities. 

Advanced in the computing are forcing us to rethink the key organizational elements and concepts held within the frameworks.  It is also forcing us to consider the relationships with other traditionally distinct tools involved in workflow and workload management, along with the actual workflow stages outside production.  Since framework-related software sits between the experimental HEP community (users that set requirements) and computational resources (dictators of constraints), changes are driven by using what is perceived to be available and providing predicted need. 

Key aspects of the frameworks are outlined in the event processing framework section of the community white paper.  This workshop provides a follow-up to the ideas formed in the that paper and to the March Naples workshop. 

The goal of this workshop is to 

  • discuss unique changes that are now being made to software
  • discuss starting or ongoing research addressing upcoming intermediate and long-term issues
  • understand intermediate and long-term concerns that the community has and wants addressed
  • discuss where similarities lie and set goals for a next meeting

At this workshop, we will bring together leading developers of framework components to address issues regarding:

  • Data model - how are they evolving to meet experimental needs now and heading towards run-4?
  • Algorithm organization - are specialized hardware trends changing what algorithm does and how it is scheduled?
  • I/O - are changes storage systems and networking affecting how data is organized and represented outside the application?

We also want to discuss how the lines between outside systems and workflows ought to be blurred:

  • analysis - can systems be coupled to provide real-time explorations not possible in the past?
  • workflow - can framework application is more direct participants in the execution of higher-level distributed (and parallel?) workflows? 
  • Alexandre Sousa
  • Andrei Gheata
  • Antoni Shtipliyski
  • Attila Krasznahorkay
  • Benjamin Morgan
  • Charles Leggett
  • Chris Jones
  • Frank Winklmeier
  • Graeme Stewart
  • Holger Schulz
  • Illya Shapoval
  • Kyle Knoepfel
  • Marco Clemencic
  • Mihaela Gheata
  • Mohammad Al-Turany
  • Paolo Calafiura
  • Scott Snyder
  • Sebastien Binet
    • 1
    • 2
      Speaker: Peter Hristov (CERN)
    • 3
      DUNE Software Needs / ART
      Speakers: Kyle Knoepfel, Kyle Knoepfel (Fermi National Accelerator Laboratory)
    • 4
      NOvA / DUNE
      Speakers: Alexandre Sousa (University of Cincinnati), Prof. Alexandre Sousa (University of Cincinnati), Prof. Alexandre Sousa (University of Cincinnati (US))
    • 3:30 PM
      coffee break
    • 5
      Speakers: Holger Schulz, James Kowalkowski (Fermi National Accelerator Lab. (US)), Marc Paterno (Fermilab), Marc Paterno (Fermi National Accelerator Laboratory (FNAL)), Paolo Calafiura (University of California Berkeley (US))
    • 6
      CMS: Integrating TBB Tasks and Accelerator
      Speaker: Christopher Jones (Fermi National Accelerator Lab. (US))
    • 7
      Roundtable Discussion
    • 8
      Closeout / Planning