IT-protoDUNE coordination (Single Phase and Double Phase)
# IT ProtoDDUNE 30 Aug 2018
## Attendants
Room: Xavi, Maxim, Cristi, Ignacio, Steve, Ken, Igor
Vidyo: Elisabetta
Phone: Stuart Fuess
## Coordination update
- Nothing to report
- Next meeting on the 13th of September
## Round table
### GPU usage
- (Xavi) Used inside the tech-lab environment. There was a ticket related to
these machines being in the critical path for data taking
-- Albert de Roig, Neil Whitehead
- (Maxim). Shouldn't be critical, need to be discussed with Double Phase
- (Elis). Never heard of it
### Maxim
- 3 different types of payload right now
-- Purity of Argon
-- Monitor for histogram/Channels vs time
-- Cosmic Ray tagger
- Will have a 3D display
- Missing beam instrumentation
- Changing the description of menus/panels for visualization generation to a JSON
based description
- Right now running a 25h queue, but could be increased (following recommendations)
- Still having some big fluctuations with transfers, but the data still needs
to be analyzed properly.
-- (Xavi) Fluctuations are kind of normal, it would be useful to be able to match
the issue to specific nodes because it may be caused by defective nodes
Cristi/Xavi: You can use xrootd to read on remote (streaming) or read specific
data sections to avoid read whole files
(Stu) We have some information regarding specific filenames that can be used to further
investigate the network issues
### Elisabetta
- System is complete, 6 last machines were installed and are being configured right now.
- Two Data challenges were run, the result was very good and now CERN FTS will be use
to move files from local EOS to the IT-CC
- There were several power cut due to cooling failure (Temp. too high in HS1?). Geoff corroborate this fact.
Q(Xavi) Do you have any plans for cosmics run in Autumn?
A: Plan to close and start the run with cosmics by the end of the year
### Stuart
New tape robot commissionned (140TB, and more coming). Currently catching up on the data.
First of the LTO robot, supposed to be 3x faster than the old robot.
Still missing a couple lines 10GB to get to 40Gb in the network setup.
There will be a request.
LTO issues related with the media3
### Ken
Good results running version 2.0
Latest release of compression (decoder seems to be right), about half a second for event
currently running on whatever data gets to the DAQ
(Maxim): Issues with the compression because it changed the data packing so it's not
clear how effective the compression is being.
(Ken) Last factor heard for compression was ~2.7. (Maxim): Should get to 4
### Igor
Beam instrumentation data flows smoothly all the way to the database (6 level of redundancy)
(Maxim). We are using this data but it needs additional because the use case is different
than the originally intended, so displays and exploitation needs additional work (WIP).
### Geoff
Q(Elisabetta). Do you know the reason for the cooling failures (HSE1)?
A(G). It was a piping issue (big pipe, other systems affected), was fixed yesterday's
afternoon.
- Filling might end by the end of next week.
- Need to clean liquid argon. Recirculation takes ~5days for the envire volume, but
don't know right now how many recirculations will be needed. Should be possible to see
some cosmics.
- Tuning the beam as we speak. Needs to pass through the cryostat.
- On the DAQ front, bad week with the cooling issues. A couple of extra glitches today
being worked on. Compression is working (~4 factor).
Tuning the firmware from the FEMBs to read 1-2,1-2, so two receivers get the exact
same data and can be correlated -> No errors detected
### IT-DB
https://cern.service-now.com/service-portal/view-request.do?n=RQF1093183