IT-protoDUNE coordination (Single Phase and Double Phase)
- Present: Geoff, Borja (IT-CM), Cristi (IT-ST), Xavi
- Remote: Steven, Kevin, Stu, Maxim
- Excused: Ignacio Coterillo
- Xavi ask for news about data rates and compression:
- (Geoff) Compression running at x2.7 and data acquisition window reduced from 5ns to 3.5ns although data taking will start without compression.
- (Steven) Data flow from EHN1 to IT-CC expected to be around 2.5GB/s
- Xavi ask for networking status after the latest results routing through the LHCOPN where some bottlenecks where identified on dCache(FNAL)
- (Stu) New pool nodes on dCache at FNAL with new network parameters, need to be tested.
- Stu reported the need to add authentication on the DAQ buffer (Kerberos). It is behind the firewall right now but want to have Auth before the start. Agreed to submit a ticket that will be followed.
- Maxim reported that single job stage-in speed varies heavily, from 100MB/s to few MB/s for 8GB files.
- (Xavi) I/O on batch nodes can vary. Hypervisors host several VMs potentially running several jobs hence local disk/network resources are shared. Suggest to explore better ways to run this jobs and avoid copy-to-the-node (stage-in) in favour of remote-reading. xrootd TTreeCache parameters can be tuned to optimise I/O, ie. the job can start running as long as first bytes come and then xroot can pre-fetch according to the I/O patterns populating a “cache” enough to give close to local performance but not downloading the entire file. This can also save substantial CPU time as the job does not have to wait until the 8GB file is fully downloaded. To be followed up.
- Xavi asked for a review of available resources on disk in preparation fo the data taking.
- (Cristi) Will have a look.
- Kevin and Geoff reported the need to feed the Grafana protodune monitoring page with the input from network stats coming from EHN1/DAQ routers and switches nowadays collected by spectrum (IT-CS)
- After the meeting Borja (IT-CM), Geoff and Xavi get together with Jerome (IT-CS) to evaluate/organise this.
- Next meeting the 30th of August
- OB:
- Steven will be at CERN from 31st of August to the 7th of September
- Discussion for CASTOR (tape) final settings to happen in the following days: data rates, data pools (CDR for datataking) and data organisation (evaluate convenience to have a dedicated tape family to ease future recalls).
There are minutes attached to this event.
Show them.
