# 24th International Conference on Computing in High Energy & Nuclear Physics

Nov 4 – 8, 2019

## Real-time HEP analysis with funcX, a high-performance platform for function as a service

Nov 5, 2019, 11:00 AM
15m

### Riverbank R7

Oral Track 7 – Facilities, Clouds and Containers

### Speaker

Dr Anna Elizabeth Woodard (University of Chicago)

### Description

The traditional HEP analysis model uses successive processing steps to reduce the initial dataset to a size that permits real-time analysis. This iterative approach requires significant CPU time and storage of large intermediate datasets and may take weeks or months to complete. Low-latency, query-based analysis strategies are being developed to enable real-time analysis of primary datasets by replacing conventional nested loops over objects with native operations on hierarchically nested, columnar data. Such queries are well-suited to distributed processing using a strategy called function as a service (FaaS).

In this presentation we introduce funcX---a high-performance FaaS platform that enables intuitive, flexible, efficient, and scalable remote function execution on existing infrastructure including clouds, clusters, and supercomputers. A funcX function explicitly defines a function body and dependencies required to execute the function. FuncX allows users, interacting via a REST API, to register and then execute such functions without regard for the physical resource location or scheduler architecture on which the function is executed---an approach we refer to as serverless supercomputing.'' We show how funcX can be used to parallelize a real-world HEP analysis operating on columnar data to aggregate histograms of analysis products of interest in real time. Subtasks representing partial histograms are dispatched as funcX requests with expected runtimes of less than a second. Finally, we demonstrate efficient execution of such analyses on heterogeneous resources, including leadership-class computing facilities.

Consider for promotion No

### Primary authors

Yadu Babuji (University of Chicago) Dr Ben Blaiszik (University of Chicago) Dr Kyle Chard (University of Chicago) Dr Ryan Chard (Argonne National Laboratory) Prof. Ian Foster (University of Chicago) Dr Zhuozhao Li (University of Chicago) Tyler Skluzacek (University of Chicago) Dr Ana Trisovic (University of Chicago) Dr Anna Elizabeth Woodard (University of Chicago) Daniel S. Katz (University of Illinois)