JCOP FWWG 16 June: NextGenArchiver

→ Europe/Zurich
Vidyo

JCOP FWWG Meeting, 16 June 2020

Indico event: https://indico.cern.ch/event/918754/

Meeting held over Vidyo teleconference

Participants (as shown in the Vidyo participants list): Anthony Hennessey, clara, cpires, Frank, FVR, G. Thomasm Jean-Charles Tournier, khomout, Luis, mariusz, Mateusz, Patricia Mendez, Paris Moschovakos, Peter Bond, Peter Chochula, pgv, Piotr Golonka, Rafal Kulagam Riku-Pekka Silvola, schlenk, Wassef Karimeh

  • News about WinCC OA:
    • patch P020 made available by ETM and repackaged for Linux ans Windows as version 20200612; testing has only started now;
    • we expect a "hotfix" from ETM to properly support Oracle instantclient 19 on Linux - this should be available by the end of this week; we will package it as a CERN patch and include in the RPM distribution
    • we encourage interested users to help in testing of this version already now and hence contribute to its validation
  • News  on the JCOP Framework:
    • second beta of the JCOP Framework 8.4.1. undergoes internal testing and a public beta should become available shortly. We aim for final release by the end of this week
    • the release consolidates numerous bugfixes
    • it delivers the newest fwFSM with numerous improvements and fixes
    • fwDIP (8.5.0) is based on the latest DIP libraries, with support for variable-length arrays
  • Reminder about the agenda: the topic of the New Alarm Screen will be presented in the next JCOP FWWG (30 June), together with the status update of the fwTrending consolidation

 

Main topic: Next Generation Archiver: status and the first public CERN release

  • Questions 
    • ALICE: expressed their gratitude for making this development reach the current stage; are the schema upgrades easy and transparent really? It depends on the actual history of the schema; based on the experience with failed upgrade due to some customization one cannot guarantee the outcome. Test upgrades should lower the likelihood of a failure (even though the failed upgrade in December was tested). ALICE will go this way
    • ALICE: upgrading the existing NGA installations? There will be a bit of additional steps (uninstalling the currently deployed NGA), still should be quite straightforward
    • ALICE: How much Influx is ready for production? IT/DB is currently not considering the opensource version as production-grade. The strategy is to keep Oracle for critical data and archive in parallel to Influx, with the latter providing performance and features such as Grafana dashboards.
      • FVR: clarifies that establishing the SLA with IT/DB is in the roadmap for the project for this year
      • Rafal explained that InfluxDB 2.0 is comming, with no data migration path... Hence some of the use cases may not be addressed; still it has its own strengths
      • FVR: we need to see what InfluxDB is capable of (e.g. scalability, applicability limits)
    • Stefan (ATLAS): why do we seem disqualify InfluxDB already at the start even though we put enormous efforts in testing it already? investments need to be protected!
      • FVR: we need to check again with IT, as there is growing interest in the use of InfluxDB across CERN. Performance figures in setups with high cardinality also need to be assessed in a systematic way. 
    • Other backends?
      • ETM collaborating with some Russian company to develop a backend for Postgress and SqlServer. The former could eventually become a realistic alternative to Oracle, provided SLA could be agreed on with IT/DB.
      • Big data backends? Apache Kudu evaluation showed very promising results last year. However investigation/development was put on hold in order to finalize the Oracle backend. We plan to resume this activity, namely when ETM will take over the Oracle backend (this will happen once we made sure all necessary features are there). 
    • Stefan (ATLAS): are there othere releases of Oracle schemas in plans?  The release on NGA in December 2020 will almost certainly have a new schema to enable some of new features. That is why an immediate upgrade of all schemas is currently not recommended.
      • is the current 8.21 schema in any way better than 8.9? The obvious one is that it enables NGA, yet still not allowing to have a DP archived into 2 ORACLE groups with different smoothing in the same schema. Still, parallel archiving to Influx already possible with 8.21
    • Stefan (ATLAS): QA looks quite impressive; yet would be good to review with experiments about the "known issues" that used to be a problem in the past. Actively seek feedback from PEOPLE WITH OPERATIONAL EXPERIENCE of RDB (e.g. details of how disconnection problems were actually triggered - switch or firewall problems, etc), and RDB used to behave very differently. ACTION: Set up a meeting
    • Stefan (ATLAS): error handling and reporting - is there anything  better than the log? Each backend has a simple diagnostic DPE with last timestamp, last error, db connection lost; in general, should be much better than in the case of RDB Archiver.
    • Stefan (ATLAS): performance in dist systems - would be good to know the actual scale of the tests; more reports will come for the next release. For the time being the performance is comparable. There are some corner cases for very large systems that will be made simpler (architectural problem being worked on)
    • J-C T. testing in Redu systems? Some of redundancy transiton scenarios already in the test catalogue. At this moment not being looked at, but planned for the release in October
    • should we include NGA in the CERN installation? Would it make sense to have it in the Framework distribution? The already existing component is 70 MB in size, and there will be frequent bugfix releases. This may be related to dependencies on ETM's patches and patches...

AOB:

  • None.

Next Meeting: 30 June, 14H00 over vidyo. Topic: status of fwTrending consolidation and the new Alarm Screen.

There are minutes attached to this event. Show them.
    • 14:00 → 14:10
      News about WinCC OA and JCOP Framework 10m
    • 14:10 → 15:10
      NextGen Archiver CERN Release 1.0 1h
      Speakers: Anthony Hennessey (CERN), Mariusz Suder (CERN), Rafal Kulaga (CERN)
    • 15:10 → 15:30
      Discussion and AOB 20m