RPC weekly meeting

Monday 17 Jun 2024, 09:00 → 10:30 Europe/Zurich

5/R-026 (CERN)

Davide Boscherini (Universita e INFN, Bologna (IT)), Mohamed Zaazoua (University of Science and Technology of China (CN))

Weekly meeting about the RPC system activity

09:00 → 09:10 Communications to/from the gas group

Speakers: Beatrice Mandelli (CERN), Enrico Pastori (INFN e Universita Roma Tor Vergata (IT)), Gianluca Rigoletti (CERN), Kacper Kapusniak (CERN), Roberto Guida (CERN)

09:10 → 09:20 Summary of TS1 2024

Speakers: Claudio Luci (Sapienza Universita e INFN, Roma I (IT)), Davide Boscherini (Universita e INFN, Bologna (IT)), Enrico Pastori (INFN e Universita Roma Tor Vergata (IT)), Enrico Tusi (INFN e Universita Roma Tor Vergata (IT)), Eric Ballabene (University and INFN, Bologna (IT)), Gianluca Rigoletti (CERN), Giulio Aielli (INFN e Universita Roma Tor Vergata (IT)), Kacper Kapusniak (CERN), Mr Mohamed Zaazoua (University of Science and Technology of China (CN))

List of interventions- mzaazoua - June 10.xlsx

Ts1 2024 Enrico's report.pdf

Ts1 2024 Enrico's report.pptx

Cavern intervention, for the detailed Summary here see the e-log:

• BOARDS:
  • LV board with increasing temperature replaced, temperature now OK.
  • LV board for Igap for sectors 11 and 12 power cycled, now OK.
    
    
• Gas gap disconnection:
  
  • Gas gaps disconnected from HV or HV+gas.
    
    
• HV problem
  • BOL.A3.05.CO.Ly1 HV channel trips due to BOL5A05.Ly1 chamber (the one that might have caused the many DAQ resyncs e-log).
    
    • After debugging on both rack and chamber sides, a problem is found on the cable powering BOL5A05.Ly1 chamber.
    • Built a splitter from the HV spider powering the BOL6A05.Ly1 chamber, but still trips due to electrical problems (checked that is not due to gas gaps).
    • Serial number of the cable did not match due to swap between layers (the same serial number on the cable rack side is Ly1 but on the spider side is Ly0. 
    • In the end, BOL5A05.Ly1 disconnected (4 gas gaps) at spider level, before cavern intervention disconnected rack-side. To be followed up by Bologna technicians.
  • BOF5C12.PI.Ly0 (4 gas gaps) disconnected from primary cable due to channel trip (DZ1 chamber driving high current at 7kV tripping the channel, DZ2 also driving high current). This chamber was connected before TS1.
    
    

• LV problem:

• • BMS1C04.PI Vee trips at 6V, now fixed.
  • BML1C03.PI Vee and Vpd OFF, fixed by board parameters reloading.
  • BML3A11. CO. DP1LY0. DZ2 connector rebuilt.
  • Canbus connector on the pad located on BOF3A14-confirm-toward sector-15 broken inside, replaced it.
  • Vpad OFF for sectors 9 and 10 during TS1 due to 48PS failure, fixed.
    
    
• Disconnection moved from rack side to chamber side for the following gas gaps:
  • BMS4A10.PI.Ly1 chamber driving high and instable current. Not possible to remove the overcurrent disconnecting DZ1 gaps, left disconnected rack-side. To be followed up by Bologna Technicians.
  • BOL.B.09.CO.Ly0.RO HV reconnected on rack-side and BOL1A09 gaps (DZ1 and DZ2) disconnected on chamber side.
  • Disconnection moved to chamber side for gas gap BOS5A10.CO.DP2.Ly0.DZ2 (difficul to reach, was skipped before). BOS5A10.CO.Ly0 chamber reconnected rack-side in HV channel BOS.A3.10.CO.Ly0.
  • SPC4A04.CO.DP1.Ly1 and SPC4A04.CO.DP2.Ly1 giving a trigger hole reconnected to the HV (no Igap readout).
    
    
• Some gas gaps repaired from gas leaks [Dubna team].

   
• Missing hits in some trigger sectors in the feet regions, powering checks:
  

  • BOF5C12.PI (s21_t4) Ly0: reconnected primary cable and one gas gap found disconnected and powered on but after some time trip on the HV channel, disconnected again.
  • BOF3A12.PI (s54_t2) Ly0: reconnected one gas gap found disconnected and powered on but after some time trip on the HV channel, disconnected again.
  • BOF5A14.PI (s57_t4) Ly0 and Ly1 missing hits: powering OK, everything seemed connected.
    
    
• Next top-priority intervention: 
  • Fix a LV Vee connector (no current for BOS5A02.CO Vee, with Vee powering FEs for 8 gas gaps, also causing BOS5A02 Vpd in overcurrent, after Friday intervention).
• See also the summary of cavern intervention for RPC trigger from Claudio [e-log]
  
  
• Communication issue on 2 mainframes, configuration rebooted [e-log]

 

09:20 → 09:35 Leaks invisigation & repair

Conveners: Dmitri Kharchenko (Joint Institute for Nuclear Research (JINR)), Enrico Pastori (INFN e Universita Roma Tor Vergata (IT)), Ivan Troeglazov (Joint Institute for Nuclear Research (RU)), Vladimir Elkin (Joint Institute for Nuclear Research (RU))

The detailed repair report from the Davide and Sinem list of trigger holes (BMS1A04 wasn't in list, but looks like trigger hole):

1. BML2A01.CO.Ly1.DZ1  Dp1_1 (inserto tubo)
2. BML2A01.PI.Ly0.DZ1  Dp1_2 (inserto tubo)  Dp2_2 (inserto tubo)
3. BML2A01.PI.Ly1.DZ1  Dp1_2 (inserto tubo)  Dp2_2 (inserto tubo)
4. *BMS1A04.CO.Ly0.DZ2  Dp1_1 (inserto tubo)  Dp2_1 (inserto tubo)
5. *BMS1A04.CO.Ly1.DZ2  Dp1_1 Dp2_1 HV connected
6. *BMS4C10.PI.Ly1 broken pipe in output
7. BOL2A03.CO.Ly0.DZ2  Dp1_1 Dp1_2  HV connected DP2
8. BML1C13.CO.Ly1.DZ2  Dp1_1 Dp2_2 (inserto tubo)
9. BML1C13.CO.Ly0.DZ2  Dp1_1(inserto tubo) Dp2_1 (inserto tubo) Dp2_2 (inserto tubo)
10. BML1C13.CO.Ly1.DZ1  Dp1_1(inserto tubo) Dp1_2 HV was not reconnected on DP2 ( foam prevent, see photo) 

SPS4A04.CO.Ly0+Ly1 - no leak, connected

BOS5A04.Ly0.DZ2 -  no leak, connected

BOS5A04.Ly1.DZ2 -  no leak, connected, HV DP1 was off

BMS6C02.PI.Ly1.DZ2  Dp2_2 (crack)  - OX, reversed

BMS6A02.CO.Ly0.DZ2  Dp1_1 leak only DP2 (reduced to 0.025 mbar/s)    - OX, resistor

BMS6A02.PI.Ly1.DZ2  Dp1_1 leak 0.03 mbar/s    - OX, reversed

BML2C05.PI.Ly0.DZz  leak 0.11 mbar/s, all corners foamed - OX

BOS4A02.CO.Ly1.DZ1  - OX, reversed

BOS4A02.CO.Ly1.DZ2  - OX, reversed

BOF5C12.PI.Ly0.DZ1  - leak 0.1 mbar/s in DP2, difficult to identify  - XX,  HV disconnected

BOF5C12.PI.Ly1.DZ1  -small leak (the same was in March) 0.02 mbar/s - OO

BOL6A01.CO.Ly1.DZ2  HV disconnected (it has gas leak )

BOL4A01.CO.DP1.Ly1.DZ1 leak 0.3 mbar/s -  HV disconnected (DP2 HV was already off)

 

09:35 → 09:45 Detector/LVL1 status

Speakers: Claudio Luci (Sapienza Universita e INFN, Roma I (IT)), Enrico Tusi (INFN e Universita Roma Tor Vergata (IT)), Eric Ballabene (University and INFN, Bologna (IT)), Gian Luigi Alberghi (Universita e INFN, Bologna (IT)), Dr Gianluca Bianco (Universita e INFN, Bologna (IT)), Giulio Aielli (INFN e Universita Roma Tor Vergata (IT)), Marco Sessa (INFN e Universita Roma Tor Vergata (IT)), Mr Mohamed Zaazoua (University of Science and Technology of China (CN)), Sabrina Perrella (Sapienza Universita e INFN, Roma I (IT)), Sinem Simsek (Istinye University (TR))

20240617_165242.mp4

Screenshot 2024-06-17 at 12.43.54.png

BOL.B.09.CO.Ly0.RO HV channel responsible for the noise problem. See the detailed e-log here.

• How the problem looks like
  
  The BOL.B.09.CO.Ly0.RO HV channel gives noise which can be seen as high rates in sectors 8, 9, 10 in both A-side and C-side in the DCS DDC panel. This noise also gets triggered by TGCs and sTGCs.
  To see the rates for the debugging, it is needed to ramp up the HV channel to 8800V in standby and wait one/two minutes to see the updated values in the DDC when the HV channel has finished to ramp up.
  The problematic rates are in the DDC L1 High pT, not in the Low pT. We found that there are non negligible rates in the Low pT but these don't give the noise.
  
  Attached there is a picture  about how the DDC panel looks like when the noise issue is detected.
  
  
• Cable connectors
  
  Attached there is a video of the badly connectors that are found in the cavern.
  
  We cannot leave connectors that way in the cavern, loose connectors may create sparkings which cause noise which gets triggered and ATLAS does not operate efficently.
  However we didn't find any visible sparking in the connectors.
  Currently we don't know yet if the badly connectors were responsible for the noise issue. Actually we checked that when the connectors are properly plugged, the noise issue is still present, suggesting not being cable connectors badly unplugged. At the beginning we thought it was just a chamber connector unplugged, the one powering BOL1A09 rack-side, but with the persistent noise we later found out that all connectors were badly plugged. Further debbugging is therefore needed.
  
  
• Other observations
  
  The power supply current does not match the sum of the Igaps.
  Message from Stefan Schlenker: the issue is not fully understood in the sense that the HV distribution behind the channel output needs still to be debugged and it would be interesting to understand the mechanism behind the (massive) noise propagation. The follow up can now be done by the RPC expert team in more calm.
  It would be good though if @Alessandro Polini could make somehow sure that this channel cannot be ramped up anymore easily in normal operations to avoid that at some point a shift crew is falling into the noise trap.

 

09:45 → 09:55 Resin update

Speakers: Ahmet Renklioglu (Istinye University (TR)), Dr Aydin Ozbey (Istanbul University-Cerrahpasa (TR))