FuncX: High Performance Function as a Service for Science

• Ryan Chard (Agronne National Laboratory)
• Anna Woodward (University of Chicago)

Room: 40/R-B10 There is a growing need to support the execution of short-duration tasks on High Performance Computing (HPC) infrastructure. To address this need we have developed FuncX: a function serving platform that is designed to provide on-demand execution of stateless functions directly from existing HPC infrastructure. FuncX aims to remove the barriers that exist between functions and large-scale computing infrastructure by simplify execution of function-based workloads, enabling integration of HPC capabilities in applications and analysis pipelines, and ultimately increasing the usability of computing infrastructure. FuncX uses the Parsl scripting library to provide low-latency, scalable, and reliable execution of tasks across heterogeneous computing environments. It securely isolates function executions within containerized execution environments and leverages Globus, as an identity and access management framework and to automate the movement of functions and data. In this talk I will describe the use cases that have motivated the development of FuncX, present the design of the FuncX execution framework, and present the prototype implementation and early experiences using the system.

Discussion Room: 40/R-B10