QPS reliability studies with CSL - progress meetings 2024

1 Jan 2024, 14:00 → 30 Dec 2024, 16:00 Europe/Zurich

Friday 22 March

15:30 → 15:50 QPS progress meeting - 1

Speaker: David Westermann (Universitaet Stuttgart (DE))

• david's questions
  • there are two sets of tabs
  • each pdsu gets a separate voltage table
  • all three get the quench
  • worst
    • pa3 & pa1
    • always two layers
  • dqhsu
    • dipole magnets
  • uqds
    • few cards in nqps in lhc will run uqds
• laure and adam
  • filtering of noise mainly
    • how does it go ok with threshold
    • experience from magnet test benches
      • power converters rather noisy
    • multiple magnets have more inductance/load and get less noise;
    • will only see from string the signal to noise ratio
      • 600 Hz spikes, apply non-linear filter
    • cannot realistically say noise
  • laure: for noise filtering block - do you have an idea of filter structure
    • yes, standard filter structure
    • shows example from uqds firmware documentation
    • spikes up to 1ms can be absorbed
• csl wants to split argument in sub-parts
  • old argument - giving overview of whole MPS
  • QPS - different components and magnets
  • QPS again, but for inner triplet
  • fmea & fta comparison
• fmea vs csl
  • hardware - do fmea & fta
  • maybe try fmecdma in socrates
  • level & side thoughts are interesting to capture

Thursday 25 April

16:00 → 17:00 QPS progress meeting - 2

Clarification of questions

Speakers: David Westermann (Universitaet Stuttgart (DE)), Laure Millet

• laure: developed argument - re-started from top-down - focus on inner triplet; defeaters added
• review and check which are to focus on
• david's questions
  1. 1. How does the UQDS react upon loss of power (assuming all the UPS etc. fail)? Will it open the quench loop? Will it fire the CLIQs/QHDSs?
        1. will open quench loop --> fast power abort
        2. units powered in redundant way, in case of full power loss of both supplies, cannot fire heaters
        3. internal power on uqds level and pdsu level is there as well - meant for diagnositc purposes but could interlock
        4. generally same for pdsu
        5. possiblity of simultaneous power outage and quench
           1. controls electronics of heaters cannot trigger without power
           2. supply lines for cliqs (UR) and heaters (UJ) same as pdsu () & uqds ()
     2. Regarding the PDSUs for D1 and D2,
        1. is it clear yet how they will look like?
           1. pretty much like inner triplet
           2. take out cliq monitoring
           3. keep separate for D1 & D2
           4. trigger fan out via PDSU or UQDS
              1. if link quenches and we have to fire all Ds
              2. then need pdsu for trigger
        2. will D1/D2 each have one PDSU?
        3. will these perform the same functions (that are relevant to D1 & D2) as in the IT (removing the CLIQ which does not exist in D1 & D2 and removing the link to the BIS) or will they also perform other functions?
           1. will need PDSU
     3. Are the conditions for the PDSU in the inner triplet to open the quench loop identical to the ones for requesting a beam dump to the BIS? In other words, will it request a beam dump both in case of spurious HDS/CLIQ firing as well as upon any QDS detecting a quench?
     4. clarify with Daniel if D1 and D2 important for this
  2. uqds upon spurious firing would saturate, but only if the condition remains for n miliseconds; at top energy 3 milisecond, but can be ten miliseconds
  3. pdsu can always dump beam via bis - but better to have active low
  4. think through big disturbance
     1. trigger heaters within few ms
     2. ups should survive for that time
     3. fuses? are rather slow
     4. supercaps - wanted to cover 100ms gap in AC
     5. question whether you could trigger the hds from its own power circuit

Tuesday 11 June

16:00 → 17:00 QPS progress meeting - 3

A summary of the progress made thus far in the reliability study of the magnet protection function of the UQDS for the Inner Triplet, along with a clarification of outstanding questions.

Speaker: David Westermann (Universitaet Stuttgart (DE))

UQDS - IT– Reliability Analysis – Quench protection function.pdf

UQDS - IT– Reliability Analysis – Quench protection function.pptx

Present: D.Westermann, L. Felsberger, J. Steckert, L. Millet

• two high level questions
  • why both cliq and HDS
    • low current cliq not effective
  • power distribution
    • uqds connected to two independent UPS

    • Jens (Unverified):

      sent on June 11, 2024 4:20 PM

      • https://confluence.cern.ch/display/TEE/Documentation+HL-LHC+DQHDS
• david slides
  • yes, single quench is most likely
    • symmetric more by wave of warm helium or beam losses that are symmetric
    • Q1A & A1B symmetric can never happen (it was decided)
  • cabling can be included and taps are routed in the same cable
    • come all out of IFS
    • one going o service gallery in patch panel, shuffling to A & B box
      • patch panel of uqds is being designed
    • connection of uQDS to PDSU
      • single ended, has protection
  • agreed to sit together and mark ciritcal path on schematics
  • jens agrees to go to pesimistic
    • could also be on settings
  • number of systems
    • normally one QDS per systems
    • 9436/2 maybe working, but not convinced
    • https://confluence.cern.ch/display/MPEEP/Circuit+Boards
      • scan and proposal for next meeting
  • inspection intervals
    • all interlocks tested yearly
      • cannot do during normal operation
      • pdsu measures trigger current to protective elements (can test without HDS on)
      • can inject test signal in input channel - this is how interlock test is done
    • in normal operation
      • permanent checkers for noisy or degrading signals
      • voltage tab unlikely breaking silently
    • elqa will only do it after LS
    • hard to construct case where voltage tap
    • HDS/CLIQ
      • policy for machine so far was once a year
      • to be checked how it will be in future
        • possibly staged
  • uqds bb
    • intermagnet busbars
    • also return busbar
    • two times two taps - but sometimes shared with coil protection
• laure
  • SPI
    • lagging
      • how much triple modular redundancy? jens - we do not have it, no radiation
    • leading
      • spikes below the filter
        • jens: would need assessment of signal quality, which do not exist yet
        • one source of spikes is power converter
      • long term data
        • is 10Hz
        • only on events have post mortem - few tens of KHz
    • stuff recorded for uqds
      • magnet voltages
      • derived signals
      • logic status
      • system performance indicators
      • configuration data (in LSA)
        • uqds has lots of adjustable units
        • configuration cross-check every start up of units
        • following intervention
        • different operational phases
      • consider different detection thresholds on different detection strategies
        • is possible - is it a question of reliability or optimizing selectivity
      • Laure would appreciate list of indicators that are monitored
        • Jens: system is designed on the nervous edge and indicates everything with a false dump
        • outside commissioning rare to see something
        • power supplies will get monitoring

Sent links

• https://edms.cern.ch/ui/file/2723589/1/Inner_Triplet_Protection_Note_docx_cpdf.pdf
• https://confluence.cern.ch/display/TEE/Documentation+HL-LHC+DQHDS
• https://edms.cern.ch/ui/file/2822048/1/EDA-04198-V4-2_sch.pdf
• https://indico.cern.ch/event/227895/contributions/477327/attachments/375657/522620/P5-4-CMJ-MPP_Workshop_Mar2013_Electrical_Distribution_VChareyre_-revMJ20131028_Corrected.pdf
• https://confluence.cern.ch/display/MPEEP/Circuit+Boards

 

Thursday 20 June

10:30 → 11:30 Discussion about the UQDS/PDSU schematics

• last UQDS version --> trigger separately
• input stage
  • more resistors for voltage withstand
  • short in voltage tab --> inductive heating ..> burns short away
  • ground floating
    • inductive via PS
    • digital via coupler
  • voltage protection
    • FETs stop - current limiter
    • blind failure potential
  • from top comes gain control
  • test signal as soon as differental converter
  • differential for ADC
    • reduces signal to noise
  • high voltages
    • from magnet - shortly
    • ESD - test by not caring in lab about it
      • there were families of insulators who had problems
  • exchange
    • monitor via busy signal
    • state machine to check if busy is not ready
    • if too long not heard back --> safe signal
    • also check if noise is still present --> otherwise trip
  • flash memeory
    • calibration of channel
      • offset (not important)
      • gain error
    • but communicated to high level software
  • voltage reference - failure--> wrong data
  • in principle
    • separate power etc
    • poweer pins shared
    • steckverbinder geshared
• midplane
  • lots of plugs
  • diagnostic not present
  • most connectors for input channels (16 x 5)
• trigger
  • all options, maybe not all mounted
  • 4 current loops
    • optionally source
  • up to 8 hds/pdsu triggers
    • depletion mode FET
      • safe against short pulling down 15V line
    • because triggering needs to be active
  • contact problems?
    • contact line across first and last pin
    • because IT won't trigger often enough to see
  • sync
    • tell post mortem to synchronize
    • 10 uQDS
      • one detects, tells others
    • off topic - force A/B
      • boards swap sending
      • tell controller to stop toggling
      • for continous logging
  • fpga not directly driving, hence buffers (in middle)
  • vdp
    • fpga
      • no clock
        • communcation still there, but only static function
        • no protection
        • clk err has monitoring --> triggers FPA via FPGA
        • currenlty not foressen, but may be added
    • debug interface
      • either spi or uart
      • uART directly to fpga
      • spi via uC
        • spi also to uart
    • uC
      • connection to fpga only via handful of pins
      • all settings go to fpga
      • can write thresholds at any time
      • reads all registers periodically and writes out
        • track record is there
        • SPI for Laure?
        • SIS could do periodical check?
      • configuration management may need update
        • LSA has config
      • fpga has column parity over register map
        • if uC starts doing wild things
        • should notice parity
        • fpga has no clue what machine does
      • if uC fails safe, FPGA keeps going
    • memory
      • local configuration stored on flash
    • local PSU
      • shorts of cap - power supply goes down
      • should lead to FPA open
      • 1.7V instead of 3.3. --> fpga does whatever
      • all components can cause
      • ceramic capacitors
        • one known case at cern
• ps
  • one big ac dc (5V)
    • several dc dc
    • 3.3 v
  • adc for voltage monitoring
    • for all rails jeweils fuer a und b
  • availability vs protection trade off
    • impossible to keep fpa loop closed when current gone
  • only protection
    • if monitoring gone, could enter intermediate stage, fpga ould do weird things
• uQds as PDSU trigger when spurious triggering
  • rather on the slow side; could be adjusted to be fast enough but then we may do too many spruious triggers

Friday 12 July

16:00 → 16:20 QPS progress meeting - 4

Speaker: David Westermann (Universitaet Stuttgart (DE))

UQDS_Reliability Analysis_2.pdf

UQDS_Reliability Analysis_2.pptx

Wednesday 17 July

10:30 → 12:00 Discussion about the UQDS/PDSU schematics 2

• jens away 19/07 - 20/08, david away 20/08 - 30/08 (hand over all infos), me away 30/08 - 09/09 & 13/09 - 20/09 - 7. oct HL annual meeting "Reliability studies on PDSU and PDSU-BIS concentrator interface"
• prepare PDSU estimates and questions & models until 20/08 - until 30/8 prasenation Draft
• DSP decoupling capacitors
  • short --> would need to check what fpga does
  • fpa w/o core voltage - only via external baustein
  • if core voltage gone - no proper data out at start of fill
• check will by per fill
  • fpga lives and is configured
  • not sure whether cable to quench heater is properly sone
  • such check should cover majority of blind failures on digital platform
  • can also ask tomasz
    • is supply voltage monitored?
    • could detect lots of faults
• blind failures
  • active triggers
  • current loops not really
  • channels not really
    • almost all safe
• send fmeca templates
• v3 - fpga card ready -
  • pdpack v2 - is the same for pdsu - tomasz
  • channel in autumn (but no big changes) - very similar to pdsu cliq - very different for hds - jens
    • re-used for big CONS in LHC
    • but send current
  • trigger (very different pdsu) - jens
    • version exists
  • midplane
• blind failures during yearly tests - no consistent record