Speaker
Dr
David Evans
(FERMILAB)
Description
The Shahkar Runtime Execution Environment Kit (ShREEK) is a threaded workflow
execution tool designed to run and intelligently manage arbitrary task workflows
within a batch job. The Kit consists of three main components, an executor that runs
tasks, a control point system to allow reordering of the workflow during execution
and a thread based pluggable monitoring framework that offers both event driven and
periodic monitoring. Developed specifically to address the challenges of running High
Energy Physics processing jobs in complex workflow arrangements, with highly varied
monitoring needs, the ShREEK toolkit is in use at multiple HEP experiments, and can
be adapted for a variety of other uses such as wrapping batch jobs to provide
detailed interactive monitoring for administrators and users alike. In this
presentation we will discuss the architecture of the ShReek system and the experience
using it in several experiment workflows.
Primary authors
Anzar Afaq
(FERMILAB)
Dr
David Evans
(FERMILAB)
