JCOP Framework Working Group Meeting
9 March 2021
INDICO: https://indico.cern.ch/event/1009822/
Participants: P.Golonka, A.Hennessey, R-P.Silvola, A.Kurepin, C.Gaspar, C.Pires, F.Varela Rodriguez, F.Glege, G. Thomas, JM Pabol Rodriguez, LuisGC, L.Goralczyk, M.Suder, M.Lechman, O.Pinazza, P.Moschovakos, P.Chochula, PGV, P.Nikiel, R.Kulaga, S.Schlenker, T.Angelova, W.Karimeh, S.Khomoutnikov
NEWS:
- WinCC OA 3.16 patch 20210112 available, containing latest ETM patches P024 and P025; more details on the downloads page; no recommendation yet
- No particular news on the JCOP Framework
- WinCC OA 3.18 evaluation and framework porting: put on hold in spite of good advancement (major part of the JCOP Framework ported); awaiting an update of the LHC Schedule expected to be released in March, as well as more news about Centos8 successor
NGA
- Presentation given by Anthony, Mariusz and Rafal is available on the INDICO page
- question to have the topic of meta-data sync expanded; what does "long" mean - large systems were taking hours to synchronize for the first time; it is as long as it is for the RDB Archiver anyway - the slowness is due to the DB Schema architecture. It does not take longer than RDB - covered more further in the presentation.
- Suggestion for test: destabilizing, breaking connection, etc and see how the system recovers: a lot of this is already being done, and in automatic ways
- Surprising that the max rate limits for RDB and NGA/Oracle is the same while the CPU resource consumptions differ; explained that it is clealy the DB Schema in the PL/SQL code which is very complex;
- Did you try to change the DB instance (different server) - no, but we used a production-grade DB.
- A performance test with "test backend" (that discards all data) is planned to stress test the messaging.
- One may change the IO/CPU capabilities on the client too; is it 100% of the performance of CPU by the NGA process causing the problem? Would be nice to see/confirm the origin of the performance bottleneck.
- Note that the NGA is multi-threaded, hence no cap on 100% of load. Would be nice to see how multi-threading works and see the 200%, etc.
- CMS: Does it mean that the standard RDB schema is used by the NGA? Yes; and we do ack that it is the frustrating limiting factor.
- how about ignoring the temp table and write directly to the data tables? Once the current stage of work is finished, we plan to have a look at this.
- in particular for ALARMS it is very penalizing, but it may require very deep changes
- we focus on backward-compatibility as the most important requirement for the time being; the problem is not really in writing (even if it could be done better). The real problem is in the performance of QUERIES.
- Acceptable performance means to be judged as the whole of the system: use the performance of the WinCC OA servers which are much cheaper than the resources of the DB... We need to use the resoures in reasonable way: the goal is to do the maximum we could get to!
- Exploring the performance of schema will require quite significant resources, esp. if we want to keep backwards-compatibility. Acked that it is worth investing the effort!
- Peter Chochula: ALICE is running NGA in production sucessfully, with all issues handled very smoothly and is grateful for the help and responsiveness of the NGA team
- What is the main selling point of NGA? The possibility to store data in other databases, with varying retention/granularity plus possibility to stream data (via custom backends); this, combined with the functionality of queries having the selection
- How the performance will evolve with multi-backend? Writing with 3 backends should be still good to go, accorrding to initial tests. Serving "all events" backends at peak to be looked at. The writing scenario has decoupling into separate processes
- How many additional production users for this year? Shortage of resources expected: preferred to limit the effort in supporting for this year still...
- Not sure if integration of grafana to WinCC OA UI EWO is well understood... Anyway, it is of 2nd priority and considered a "nice to have" feature. People would usually use the web-version for any non-trivial analysis - not sure if embedding in WinCC OA ui relevant.
AOB
There are minutes attached to this event.
Show them.