Speaker
Description
The ATLAS Event Service (ES) has been designed and implemented for efficient
running of ATLAS production workflows on a variety of computing platforms, ranging
from conventional Grid sites to opportunistic, often short-lived resources, such
as spot market commercial clouds, supercomputers and volunteer computing.
The Event Service architecture allows real time delivery of fine grained workloads to
running payload applications which process dispatched events or event ranges
and immediately stream the outputs to highly scalable Object Stores. Thanks to its agile
and flexible architecture the ES is currently being used by grid sites for assigning low
priority workloads to otherwise idle computing resources; similarly harvesting HPC resources
in an efficient back-fill mode; and massively scaling out to the 50-100k concurrent core
level on the Amazon spot market to efficiently utilize those transient resources for peak
production needs. Platform ports in development include ATLAS@Home (BOINC) and the
Goggle Compute Engine, and a growing number of HPC platforms.
After briefly reviewing the concept and the architecture of the Event Service, we will
report the status and experience gained in ES commissioning and production operations
on various computing platforms, and our plans for extending ES application beyond Geant4
simulation to other workflows, such as reconstruction and data analysis.
| Primary Keyword (Mandatory) | Data processing workflows and frameworks/pipelines |
|---|---|
| Secondary Keyword (Optional) | Distributed workload management |
| Tertiary Keyword (Optional) | Distributed data handling |
