Speaker
Description
Physicists performing data analyses are usually required to steer their individual, complex workflows manually, frequently involving job submission in several stages and interaction with distributed storage systems by hand. This process is not only time-consuming and error-prone, but also leads to undocumented relations between particular workloads, rendering the steering of an analysis a serious challenge, especially for newcomers to the field. In this presentation, I will demonstrate the main components of the Luigi Analysis Workflow (Law) package which is developed independently of any experiment or the language of executed code. Its core consists of flexible, pythonic workflow descriptions, interfaces to remote batch job and storage systems, as well as a granular environment sandboxing mechanism. In the second half, I will highlight the recent key changes to the package that were driven by requests of the user base that increased steadily over the past years.
