Discussion related to general stat transition plans:
Will: desired endpoint is that felixcore/felixstar is stable enough to run all the time, and maybe be occasionally reset. We have supervisor processes (our version of PMG) monitoring on FELIX. FELIX should not be visible on run control, but there should be monitoring block in run control (that can run on ttc felix hosts and data felix hosts). We think we can implement these functionalities this year.
Will: FELIX is always listening, intelligence of whether or not data is passed on happens in sw ROD.
Georges: Do you plan on having publishing to IS, especially for TTC hosts (LAr has no idea what is happening, what is received and if there is a problem on FELIX). Will: This is possible, and can be implemented. Mark: FELIXcore/star are independant on TDAQ, we have seperate process that reads out an elink, where info from different registers can be forwarded to IS or ERS. We need to configure what variables we actually want to write out. Question to LAr: What do you want written out, can you compile a list?
Alexis: is it correct that no steps on slide 6 will be needed once felixcore is stable? Will: not entirely, resets for l1 counters and some things will still need to happen, and this will be integrated in partition. This is on FELIX’s TO DO list.
Georges: On slide 7, the ToHost reset now has to be done at each config (what is procedure for stopless recovery?). This should come from TTC system knowing what value to reset from acr system. Surgei: for ttc restart, the sw rod assumes that ttc counters will be properly reset, and then the sw rod assumes it will get info that matches current state of the CTP. Will: Our idea is that we have commands that you have to send for resets for example (have hook in partition), but for things like BCR swap or standard permeant configurations, the idea is that for each felix there is config files that includes elink config etc. This info is stored and whenever server is power cycled this config is automatically restored. The idea is that LAr wont' have to worry about that stuff once the config is defined and stored in the future. For some specific things like resets, there will be hooks in partition.
Georges: For the fixed configuration, it would be nice to configure at each run instead of when felix servers power cycled. You never know what kind of funny state system can be in for each run. Mark: this is true for DAQ, but for DCS we can't keep reconfiguring since it disrupts what we are reading at the time. Maybe we can check configuration instead of reconfiguring? Will: Yes we should have checks before going into running state.
Georges: We can check when FELIX has something within the partition and test it. The most important is resetting erc, so can start with this for now.
Discussion related to debugging the particular problem at hand:
Serguei: sw rod is ignoring all data and throws it out until it is running.
Georges said that he does not think is not generating the garbage during resets/configuration since we don’t see any data with fedump, but do see errors/garbage in FELIXcore. Mark asked if we can be sure of this. Frans pointed out that FELIX will not forward data if it doesn’t start with SOP and end with EOP as mandated by the FULL mode protocol. Firmware will raise error flag if EOP does not come. It should also report CRC Errors
Mark: Could you see if problem happens with felixstar? Can swROD be run with felixstar?
Serguei: the swrod in gitlab supports netio next, but since netio next is not supported by tdaq, would have to take the swROD from gitlab. swrod taken from tdaq rel will not work. should checkout swrod package in same area where netio next resides.
Geroges: We are using TDAQ 9.1.0
Serguei & Mark: Then, it should work with tdaq 9.1.0, can run there and point to latest swrod (for single thread receiver).
Serguei: With felixcore, you can try a swrod debugging utility, the utility will subscribe to all elinks and check consistency of ttc info in data packets, if it finds any error, it dumps packet to standard output. The utility is called swrod_netio_subscriber, run with -h to see list of command line parameters