Speaker
Description
The Joint Genome Institute (JGI) is a part of the US department of energy and is serving the scientific community with access to high-throughput, high-quality sequencing, DNA synthesis, metabolomics and analysis capabilities. With ever increasing complexity of analysis workflows, and the demand burstable compute, it became necessary to be able to shift those workloads between sites. In this talk we will present JAWS, the JGI Analysis and Workflow system, which enables users to model their workflows using the Workflow Definition Language (WDL) and bring them to execution on a geographically distributed number of sites. We will discuss the architecture of JAWS, from underlying technologies, to data transfer and integration with HPC schedulers (eg Slurm). We will go into challenges encountered when running at multiple sites, among them integration and identity management and will present the status quo of our efforts.
| Speaker release | Yes |
|---|
