JCOP FWWG 19 Oct 2021: Archiving

→ Europe/Zurich
Zoom Meeting ID
61712010501
Host
Piotr Golonka
Useful links
Join via phone
Zoom URL

JCOP FWWG 19 October 2021

INDICO Link: https://indico.cern.ch/event/1085978/

Participants: pgv, Raúl Jiménez, Giovanna Lehmann, Piotr Golonka, Leonid Kopylov, Mateusz Lechman, Rafal Kulaga, Wassef Karimeh, Mariusz Suder, Stefan Schlenker, Cristina Vazquez Velez, Peter Bond, Cristina Vazquez Velez, Clara Gaspar, Jean-Charles Tournier, Fernando Varela, Christophe Pires

  • "Clever archiving" - presentation from EP/DT (Giovanna) proposing new use cases that could be addressed by the new archiver: "varying precision" and "post mortem" analysis - see slides attached
    • comment from Stefan: maybe relative deadband could already deliver on some of these cases
    • comment from Stefan: one may need to be careful when re-configuring archive parameters. Past experience with the RDB Archiver showed that there used to be some inconsistent behaviours there.
    • clarification from Rafal: the NextGen Archiver has these mechanisms completely rewritten; they will surely need to be tested even more to exclude any risk of misbehaviour
    • ATLAS strongly supports the use case allowing for "PostMortem" - it is absolutely something that many people would find useful; there are high hopes that the NGA will be able to provide it
    • Same comment from CMS; is there already any time scale at which we may expect the "flash back" (or "post mortem") feature? This would be VERY interesting for phase-II with new subdetectors that will need to be understood! 
    • Rafal clarifies that the actual archiving to two backends in parallel is already available in the NGA now; what remains is how to copy-over the data of the events...
    • It would be interesting to copy such data from the short-lived storage to yet another storage (rather than the main archive), as experts will likely find it useful to go back to it again and again - seems a pretty common case.
    • Could the list of use cases be still ammended? Definitely! Today's presentation is merely an invitation to the discussion and to formulate what is needed! 
  • Rafal reported on the progres with the NextGenArchiver, the Grafana backend for Oracle and the upgrades of schemas, and also addressed the use cases metioned by Giovanna (see slides attached in indico)
    • what are the actual plans for the schema upgrades?
      • The upgrades to 8.20-CERN1.6 version would only be applied for projects that need the NextGen Archiver; applying it to others may be considered as wasted effort, as the schema may still evolve.
      • ATLAS hoped to have the schema upgraded during the LS2, to make it possible at all to consider upgrading to NGA during a EYETS and profit from NGA before the RUN3. Upgrading a DB schema during EYETS is much more difficult to arrange - need to be agreed with DB coordinators.
      • Similar requirement signaled by LHCb: would be interesting to have e.g. the central systems upgraded to NGA 
      • Rafal and Fernando clarified that due to the manpower situation (fellow leaving, new one starting) and other tasks (selection/validation of the new backend technology) it would be better to limit the number of production deployments that need to be maintained, so that in case support is needed it could be delivered effectively. However, obviously, a couple of deployments are possible.
      • Wouldn't it be effective to start deploying and finding bugs already earlier (in systems other than ALICE), now that the development is in progress and developers are allocated (ie. before LS3)?
      • Rafal clarified that the actual change in schema between 8.9 and 8.02 is not as huge as one could expect (rather "cosmetic")
      • Fernando clarified: the intent is to homogeneize the schemas that are in version smaller than 8.9 and upgrade them to 8.9
    • CMS: What is the schema version required for the grafana plugin
      • it does not require the 8.20 schema - the standard 8.9 should work; it may probably also be made to work with the CMS customizations.
  • AOB
    • the presentation of "What's new in jcop-framework-8.4.2" postponed to the next FWWG meeting (likely in 2 weeks) to prepare more in-depth presentation of the new features of fwFSM
    • A proposal of "CTRL++ coffee" - informal technical meeting to share experience with new features of WinCC OA from developers' perspective (e.g. the use of CTRL++ or OO-panels). Not much interest signalled - proposal postponed.
There are minutes attached to this event. Show them.
    • 14:00 → 14:05
      News about WinCC OA and the JCOP Framework 5m
      Speaker: Piotr Golonka (CERN)
    • 14:05 → 14:10
      What's new in JCOP Framework 8.4.2 5m
      Speaker: Piotr Golonka (CERN)
    • 14:10 → 14:30
      Update on the status of NextGen Archiver 20m
      Speaker: Rafal Kulaga (CERN)
    • 14:30 → 15:00
      New use cases for archiving - discussion 30m
      Speaker: Giovanna Lehmann Miotto (CERN)
    • 15:00 → 15:05
      AOB 5m