BOL.B.09.CO.Ly0.RO HV channel responsible for the noise problem. See the detailed e-log here.
- How the problem looks like
The BOL.B.09.CO.Ly0.RO HV channel gives noise which can be seen as high rates in sectors 8, 9, 10 in both A-side and C-side in the DCS DDC panel. This noise also gets triggered by TGCs and sTGCs.
To see the rates for the debugging, it is needed to ramp up the HV channel to 8800V in standby and wait one/two minutes to see the updated values in the DDC when the HV channel has finished to ramp up.
The problematic rates are in the DDC L1 High pT, not in the Low pT. We found that there are non negligible rates in the Low pT but these don't give the noise.
Attached there is a picture about how the DDC panel looks like when the noise issue is detected.
- Cable connectors
Attached there is a video of the badly connectors that are found in the cavern.
We cannot leave connectors that way in the cavern, loose connectors may create sparkings which cause noise which gets triggered and ATLAS does not operate efficently.
However we didn't find any visible sparking in the connectors.
Currently we don't know yet if the badly connectors were responsible for the noise issue. Actually we checked that when the connectors are properly plugged, the noise issue is still present, suggesting not being cable connectors badly unplugged. At the beginning we thought it was just a chamber connector unplugged, the one powering BOL1A09 rack-side, but with the persistent noise we later found out that all connectors were badly plugged. Further debbugging is therefore needed.
- Other observations
The power supply current does not match the sum of the Igaps.
Message from Stefan Schlenker: the issue is not fully understood in the sense that the HV distribution behind the channel output needs still to be debugged and it would be interesting to understand the mechanism behind the (massive) noise propagation. The follow up can now be done by the RPC expert team in more calm.
It would be good though if @Alessandro Polini could make somehow sure that this channel cannot be ramped up anymore easily in normal operations to avoid that at some point a shift crew is falling into the noise trap.