- Investigating job slowness compared to other sites in LSST (RAL 3:10:12, IN2P3 1:17:53, LANCS 1:39:05)
- IO from echo?
- Current thinking from LSST is that reading files from echo takes twice as long as other sites
- 0.6s per file at RAL, over 0.3s at others. Not much but over 20,000 files adds up! (And this is a small test data set not a real one)
- LANCS mentioned today in the storage meeting they are creating a pool just for LSST with 4+2 to try and improve performance for LSST
- CPU / memory bound?
- Request to all the sites if there is a shared space at their site to create a sqLite database within a DAG (job of jobs) to track progress within the DAG
- Informed requester of infrastructure at RAL and they seem happy that the local scrath for jobs and ECHO would be fine
- Will begin movement of ComCam data to RAL this week