Comments & questions:
- Michael & Marcel: storing mapping of job ID to tasks is done via json files (for tracking finished & pending jons)
- Moritz & Michael: what about resubmitted removed/deleted jobs (outputs)
- Artur: HTCondor Clusters of jobs?
- Marcel: Not done yet, but planned ---> submission faster
- Matthias: input sandboxes are created per job ---> too many sandboxes produced. Can be solved by HTCondor clustering
- Marcel: bypassing input HTCondor sandbox: job manager, job file factory per submission creating only shell script & jdl file.
- Marcel: instead of creating a sandbox, create the environment by yourself (within a task), upload to remote SE, and download within job
- Moritz: example of complicated + large-scale law examples?
- Marcel sending it in a chat
- Matthias: available at ETP
- Michael: profiting within b2luigi & law works nicely through going upstream with parts going to luigi
- Marcel: luigi tries to keep limited as it is, law is too overwhelming to be contribution package (e.g. having storage part)
- Moritz: what about documentation:
- Marcel: wants to pushing it as soon as ready with 2 papers in the pipeline.