Login

Automated Testing of FRAS SCADA

→ Europe/Zurich
926/1-031 (CERN)

926/1-031

CERN

12
Show room on map
Automated Testing of FRAS SCADA
Zoom Meeting ID
68688303881
Host
Roberto Fernandez Bautista
Useful links
Join via phone
Zoom URL
Hide

Attendees: Ricardo, Brad, Miguel, Marcin, Roberto
Indico: https://indico.cern.ch/event/1710662/
FRAS software meeting minutes archive: https://confluence.cern.ch/spaces/FRAS/pages/264083883/Meetings+and+Minutes 

Review of causes of errors

  • FESA development ran until the last minute — no time left for SCADA-side work before testing started
  • MLStepperAxis class not currently included in the test suite
    • Simulated MLStepperAxis instances will be used instead: already discussed with Enrica (CEM/MRO)
  • Full automatic testing on the SCADA side: work ongoing now to define the development paradigm

Marcin's Presentation — "FRAS UI Testing"

  • Testing is needed, automated testing is even better — but root causes of problems often come from misunderstanding, not the testing itself
  • Problems can originate in SCADA, FESA, or communication — but always surface visibly in SCADA
  • Incomplete/unmaintained specs make it hard to verify implementation matches expected behaviour; changes are often only discussed in meetings/emails/Mattermost, so nothing is tracked
  • Need: user-facing spec (properties, dependencies, behaviour) + dev-side documentation generated directly from the implementation — otherwise it's a "telephone game" between user and developer
  • Three testing levels considered:
    • UI only — test UI in isolation in WinCC OA, assuming FESA/comms work; manual, forced field values; could use predefined test-scenario panels
    • Full stack without UI — verify SCADA datapoints reflect FESA state (dpSet/dpGet); could be integrated into the existing pipeline via the same Python process that talks to FESA
    • Full stack with UI — most complex; no current tooling for automated UI checks, and no practical way to include an operator in an automated loop
  • Building/maintaining an engineering UI test suite is a significant, ongoing effort — tests need updating whenever the implementation changes
  • Current framework lacks UI state recognition; a fix is coming "soon" but too late for FRAS timeline
  • Adopting the MVVM pattern would decouple UI from business logic, making automated assertions easier

Object Specifications

  • Main complaint: specifications are missing
  • Without knowing the framework constraints it is very hard to produce detailed layout of a panel that might turn out to be very hard to implement.
  • Specs don't need to be highly prescriptive - a textual description of what information the panel should show is enough
  • Word documents are fine as a starting point
  • We need a way to version-control changes after that. Ricardo suggests CIET repo
  • Once development started - detailed description of each FESA variable name and description is already done automatically, with documentation being auto-generated. See https://fras-cs.docs.cern.ch/
  • Action (Ricardo): send the CIET repo as a good-practice example of how this is done. Link: https://gitlab.cern.ch/ciet/documentation/-/tree/dev/SCADA?ref_type=heads

Testing Approach Agreed

  • UI testing: manual: fake input values and confirm the result matches expectations
  • Integration testing: add SCADA dpSet to the GitLab pipeline so it can set values and confirm expected behaviour
  • Pipeline ownership: SCADA pipeline to be owned by BE/ICS/STF. Any FESA changes that break it require review by owner before release (to confirm)
  • Action (Marcin/Brad) Meet to discuss how to integrate the pipeline to gitlab.
There are minutes attached to this event. Show them.