Conclusions of File Summary Record Brainstorming

It was decided to address separately the use cases requiring an external file independent of the event data file, and the use cases of storing FSR information in POOL files together with the event data

1. FSR in POOL file
There are two classes of FSR object required:

- A job summary record, that stores information about the processing step, essentially the provenance information that is written in the book-keeping by Dirac: list of input files read, number of events processed, program version etc. This record is general to all applications and should be provided and filled centrally by the framework (Action core software team)

- Specialised records for special needs (e.g. "luminosity" measurement both for real data and for simulation). These are special objects handled by specialised algorithms, to be provided by the individual projects requiring these records.

Thus there are a small number of different specialised objects to be written into/read back from FSR, with little scope for common functionality. Nevertheless these objects should be properly designed and documented. Since they are persistent objects, it is suggested to use the GaudiObjDesc tools to define these classes.

Clarification is needed, for the LumiFSR use case, on how the file ID of MDF input files is saved on the output FSR. The FSR service should be enhanced to prevent loading of duplicate records into the TES, when opening files that are ancestors of files previously opened in the same job.

2. FSR in external file
There are two classes of object required:

- Job summary records to be used by Dirac/Ganga to monitor successful job execution. The content is probably similar but non necessarily identical to the POOL job summary.

- Generic user counters. These are counters saved by user algorithms, which Ganga has to sum when combining the output of split jobs. The idea is to provide a means of combining the statistical information that is printed by each algorithm at finalize().

It was agreed to implement this file in XML, Rob will work on providing an XML schema to define the objects. A service will have to be written that can write this XML file, converting the C++ objects to XML objects.

It was also agreed that the current implementation of counters in GaudiCommon can be used to provide the persistent user counters. A property will be added to GaudiCommon to define (by name) the list of counters to be saved. The framework will automatically save to the XML file all counters flagged in this way. 

A post-processing step in Ganga will combine the counters when merging split jobs, and functionality provided to print the counters in the same format as if printed directly by GaudiCommon::finalize()

Since it is possible to inspect the Gaudi components from python, it may not be necessary to put the data to be saved on a Gaudi transient data store. gaudirun.py could collect the information and pass it to the XML file writing service when finalizing the application.

It was felt unneccessary (and even dangerous) to provide similar functionality for automatic saving to the the POOL FSR.