Choose timezone
Your profile timezone:
256th Machine Protection Panel Meeting (Injectors)
→
Europe/Zurich
874/R-018 - salle de réunion activité CCC (CERN)
,
Description
The 256th MPP will take place in room 874/R-018 (CCC glassbox) and Zoom.
External references
- 3224610
- 256
Machine Protection Panel
Zoom Meeting ID
94542110238
Description
Zoom room for the Machine Protection Panel
Host
Daniel Wollmann
Alternative host
Christoph Wiesner
Useful links
Join via phone
Zoom URL
Participants
Bettina Mikulec (BE-OP), Cedric Hernalsteens (TE-MPE), Gregory Pigny (TE-VSC), Christoph Wiesner (TE-MPE), Jan Uythoven (TE-MPE), Piotr Skowronski (BE-OP).
Follow-up on the TT60 vacuum valve interlock issue (G. Pigny)
Gregory summarized the event which happened in 2023. The fast valves in the SPS were originally installed to protect septa against fast pressure rise (closing time < 50 ms). Two types of fast valves are installed in the extraction areas of the SPS: valves with electric actuation (VVFA) and valves with electropneumatic actuation (VVFB). The VVFA valves are installed in LSS2, LSS4, LSS6 and TT60. VVFA valves are locally powered from the tunnel 230 V electrical outlets. These valves are controlled from racks located in the BA. Gregory noted that these valves are obsolete equipment without replacement candidates.
The timeline of the problematic event which occurred in 2023 is as follow. The fast valve located in TT60 (VVFFA_610213) was not powered and at the same time the clsing mechanism was triggered: both the open switch and the closed switch activated as the valve physically closed. The vacuum piquet then confirmed the issue and intervened to re-power the valve. In the meantime, several beam injections took place while the valve was closed and in error.
When the valve closes, the “closed switch” activates as the valve is triggered, however, the “open switch” remains active until the valve goes back to the “ready to open” state, which is a slower process and requires power. As the power was cut, the valve could not move to the “ready to open” state and both switches remained active.
The valve is interlocked and connected to the BIS, the acquisition chain only checks the “open switch” state.
It must be noted that these valves are not required anymore and were never used as intended. Jan commented that the valves would be likely too slow to appropriately protect the septa. Gregory also added that although the valve itself is fast, the acquisition and interlocking chain is not fast enough. The interlocking vacuum gauge is also too far from the septum to allow a fast reaction time. Daniel asked if other similar valves are installed in the complex. Gregory replied that other machines have fast valves, but these have different hardware and acquisition chain.
Gregory then presented the short-term solution which was put in place:
- A verification of the powering of the valves was enforced during commissioning from April 2023 onwards.
- The interlock cables were disconnected so that the valve would not close in case of an interlock.
- The 230 V extension cables connection and routing was improved in June 2023 in the tunnel to avoid accidental disconnection.
- During the YETS23-24 a PLC solution was implemented, which evaluates both switches and will provide the vacuum user permit to the BIS only if the “valve is open and not closed”.
The long-term solution consists simply in removing all the fast valves. This will be done during LS3.
An ECR detailing these changes is in preparation.
A Major Event Report (MER) has been prepared and contains the details of the event.
Jan asked if the MER should be reported to the IEFC. Bettina replied that the technical details should not be repeated however the recommendations should be presented as an AOB to the IEFC.
Review of the Linac 4 BCT watchdog interlocking policy for high loss events (P. Skowronski)
Piotr gave an overview of the BCT-watchdog systems in Linac4. The BCT watchdog (FESA class “BCTWD”) computes the beam losses between a pair of BCTs and the transmission for each beam pulse. If any of the two is out of a permitted range, then the watchdog enables a BIS interlock. Logic in the FESA class allows for the transmission to be occasionally out of range (relative beam loss threshold); this is referred to as “low loss watchdog interlock”. Any occurrence of out-of-range beam losses (absolute beam loss threshold) triggers the interlock.
The experience gained in operation shows that many of the high loss events are caused by RF breakdowns. As these are frequent, OP simply resets the watchdogs, and no further action is taken if the beam simply comes back, and no further high loss events happen.
Piotr then described the proposed changes. The present system will interlock all users in case of a high loss event. These events are mostly isolated.
The proposed change is to modify the specifications of the BCT watchdog FESA class such that it can allow for occasional high loss events if they happen less often than a defined threshold within a defined time window. This aims at reducing the machine downtime.
One event per 12 hours would be allowed. Only two BCTWD devices would be affected by the change (the FESA class of the others would be modified as well, but the threshold would be set such that no event is allowed).
A risk assessment was performed for different scenarios of equipment failure.
In case of a quadrupole power converter issue, the focusing will be altered and the beam would be scraped. The losses would be distributed over a sizeable area and therefore the risk of damage would be low. These power converters are monitored by the SIS which will in turn interlock the beam. Starting in 2025, the external condition is configured such that the beam with the affected destination cannot be executed. The risk for the machine is very low.
In case of a dipole power converter issue, the beam trajectory would be incorrect. The losses would be punctual and could impact a joint in a flange and create a leak. Considering the beam parameters, this appears to require several consecutive pulses impacting the same location. SY-STI is performing simulations to confirm this assumption. The proposed change would not be implemented until the simulation results confirm that multiple shots would be required before damage could occur. In addition, the dipole power converters are under surveillance of the FGC interlock, which is connected to the BIS. The risk for machine safety is very low.
In case of a dipole corrector power converter issue the losses would be distributed over a certain area. This would require multiple consecutive pulses before damage can occur. These devices are not directly monitored by any interlock system. The associated risk to damage any equipment with three bad pulses is low.
In case of an accidental change of settings, the effects would be like a power converter trip. In case of dipoles, the FGC interlock has a narrow window, therefore the setting change would be limited to a save envelope. For quadrupoles and dipole correctors there is no such protection however the risk for the machine is low (see above).
In case of field breakdowns in the cavities the beam energy will deviate from the nominal value and the subsequent focusing and bending angles will change. Unless the breakdown occurs in the last cavity the effect of incorrect focusing will distribute the losses over a large area of the vacuum chamber. In case the last cavity is affected, this will change the bending angle of the downstream dipole(s). The beam impact could be concentrated on a small area. However, the probability that the following pulse or pulses would suffer the same RF breakdown is low.
The details of the proposed implementation in FESA are available here. Two new variables will be defined for the BCTWD class: the “highLossCounterThreshold” and the “highLossInterval”. Both variables will be protected with a MCS RBAC role. It should be noted that resetting the watchdog will not reset the counters allowing for the bad pulses. These variables will initially allow for 1 event per 10 hours (or per 12 hours). This could be reviewed in the future under the MPP approval.
Bettina commented that this change needs to be tested extensively at the commissioning. Jan asked how this will be tested. Piotr replied that error cases will be created manually (change of quadrupoles or correctors settings), with the WD loss threshold lowered. Bettina asked to include the tests in the checklist tool.
Daniel concluded that the MPP endorses the proposed change, pending the confirmation from SY-STI regarding the dipole power converter failure case. The ECR is in preparation and will be distributed when the simulation results are available.
There are minutes attached to this event.
Show them.
