Choose timezone
Your profile timezone:
EDM4hep Discussion
→
Europe/Zurich
Vidyo
Vidyo
EDM4hep Live Notes
==================
Date: December 8, 2020
Indico: https://indico.cern.ch/event/983031/
This is a document for taking notes during EDM4hep meetings.
Connected: Benedikt, Joseph, Tao, Andre, Jiaheng, Thomas, Wenxing, Teng, Sang Hyun, Birgit, Gerri, Placido, Clement, Valentin, Frank, Weidong
Apologies:
## Introduction and General Points
## Progress and Discussion
## Podio
### Podio Design Discussion (Benedikt, Thomas)
* see slides on the agenda
* Parallel Input
* GG: Where is root in this? At the reader Level? How does ROOT with all its IO fit in here?
* BH: Output of ROOT are the pods, reader fills the podio::Objects
* FG: Different approaches for SIO and ROOT IO in podio. Other IOs (e.g., HDF5) might have even different approaches
* BH: Maybe mostly an issue for the "lazy" reading
* TM: Could have different serializers for different IO backends. Advantag of modular approach to swap in and out the components. User doesn't have to know the details. Could attach different unpackers to different events.
* FG: Root raw data buffers have to be uncompressed? SIO can have compressed buffers
* BH: Have to split operations on filling buffers and unpacking, and then de-serializer. For ROOT unpacking just wouldn't be done.
* Schema Evolution
* BH: Technial details in another presentation. Message of the day: Schema Evolution can be treated by the de-serialiser
* Questions
* VV: Why do we need a plugin manager?
* BH: In bigger applications not everything should be done at compile time. Switching IO backend can't be done at the compile time.
* WL: Data in the eventqeues defined by podio?
* BH: Can be podio datatypes, but also other data types
* GG: Schema Evolution: Install the yaml files to understand changes?
* BH: No, would be compile time translation between schema versions
* GG: Avoid putting too much in the yaml file? Full copy of the class schema, or how?
* BH: Not fully thought through yet. Could be full version, or classes reference base version. Should always translate to the latest schema.
* GG: Forward and backward compatibility?
* FG: Has to be decided by the individual EDM maintainers, podio providing the tools
* FG: Event, and EventStore? Will there be an event, as there is only an EventStore at the moment.
* BH: For the standalone there will be a full Event Class. For the Framework case will be lighter Event Class.
* FG: Not sure this is the best to have two different Event Classes.
* TM: Gaudi hides the event from the framework user via the data service
* VV: Extensibility of the datamodel would be good. Podio should give possibility to store other information.
* FG: Relates to metadata and generic event user data? Limiting to the event data model is a feature.
### Issues
#### more fixes to unsigned and shadowed variables
https://github.com/AIDASoft/podio/pull/158
*
#### CollectionIterator does not fully satisfy iterator concept
* https://github.com/AIDASoft/podio/issues/150
* For const collections, would be useful, e.g. using std::find on the collection
#### Idea: True reference collections
* https://github.com/AIDASoft/podio/issues/146
* remove indirection
* to test if there are performance indications
* FG: use case could be subset collections
* FG: Remove difference between reference collections and non-reference collections
* TM: not clear if collection types can be the same
* Would be very useful
* otherwise functions need to be templated; functions based on begin/end to iterators
### PRs
* https://github.com/AIDASoft/podio/pulls
#### Disable collection operator=
* https://github.com/AIDASoft/podio/pull/154
* Review: Benedikt
#### CMake handle standard args
* https://github.com/AIDASoft/podio/pull/149
* Merged
#### Make EventStore non-copyable
* https://github.com/AIDASoft/podio/pull/152
* Merged
### c++ concepts
* BH: add compile time checks for class behaviours: e.g., movable
### issue w/ ROOT and (vectors of) non-copyable collections
* happens in ROOT 6.22
* PM: there is a patch available in LCG repository
* ROOT team is working on a general solution
### What are the different branches in the root file?
* Related to use in RDataFrame
* Encode more information in the _relation_ branch names?
### Multi-Threading
See minutes of https://indico.cern.ch/event/969468/
--> Open issue(s) for further discussion
* Thomas and Benedikt discussing and started to work, nothing to report yet
#### "event class" in podio
* Currently being perceived
### Meta Data
#### Usage of "metadata" for user defined data
* need to check if current implementation addresses all use cases
* need test use-cases
### Issues following MetaData Developments
* cannot write out event data previously read from file
* Issue: https://github.com/AIDASoft/podio/issues/103
* Test: https://github.com/AIDASoft/podio/pull/102
* TM: have fix, but break framework core handling of collections
* there seems to be a fix in the Gaudi ROOTWriter in FCC-EDM !?
* historical reason, fixes and developments should be back ported to PODIO
* ...
#### Writing second file with another tree
* https://github.com/key4hep/K4FWCore/issues/10
* this problem only happens in the GAUDI framework
* if user tries to write an additional ROOT file
* to be addressed
### EventStore
### Schema Evolution
- Version for object descriptions, etc.
- Open issue: https://github.com/AIDASoft/podio/issues/86
### Features
* Subset collections?
## LCIOConverters
* k4LCIOReader:
* https://github.com/key4hep/k4LCIOReader
* Moved k4LCIOReader code to key4hep project
## EDM4hep
https://github.com/key4hep/EDM4hep/pulls
### EDM4hep tools
* Thomas, Clement:
* Where to put tools using EDM4hep? In EDM4hep or separate repository?
* VV: Would be nice to keep EDM4hep stable, so separate repository.
* FG: Concur. Many releases for LCIO just to update tools.
### Issues
#### k4FWCore: cannot read collections back
* https://github.com/key4hep/k4FWCore/issues/27
### DD4hep output plugin
--> Moving to DD4hep
#### Testing the ddg4 plugin of edm4hep
* LCG stack not working?
* https://github.com/AIDASoft/DD4hep/pull/740
* try basing on dev3/dev4, hsf installation
### Repository is getting too big
* github pages branch is getting large
* Can they be created on the fly?
* Use a different host for the website
### Tracker Hit
* Q: Tracker hit input to tracking algorithms?
* A: In LCIO different traker hits: Planar and Cylindrical using inheritance. Still needs to be adressed how to do this in EDM4hep.
* Q: Is inheritance needed?
* Q: What to use for Driftchambers?
* Open issue:
### Need review of EDM4hepDelphes output
* Output is not stored if no tracks or towers created (?)
* Allow also other types to be part of the list
* Allow construction in a more general way (VV)
* Need to assume there is a complete list of reconstructed particles?
* Relations:
* FCCSW: Issue with more than one RecoParticle pointing to one MCParticle
* In LCIO to Delphes, conversion happens later than in edm4hep, maybe delphes does de-duplication?
* TODO: open issue in edm4hep with reproducer (VV)
--> Moving to separate repository
#### Reworking output handling
- TM: better address edge cases
- TM + CH: FCC cases, not sure if delphes or output
- https://github.com/key4hep/k4SimDelphes/pull/5
- TM: How and what to test?
-
### PRs
https://github.com/key4hep/EDM4hep/pulls/
### Release 1.0
* Need:
* ~~Plugin~~
* ~~Eventheader~~
* ~~Meta Data (Event / Run Parameters)~~
## AOB
### Generator infrastructure in FCCSW moved to EDM4hep
* VV: show at next key4hep
### Framework integration of k4SimDelphes
* On hold until delphes output reworked
* Started with updating the Gaudi documentation
* New cmake in Gaudi v35
### Dual Read-out calorimeter for FCC
* Special data structure used for that simulation
* Tried to use edm4hep, some issues
* Present in a future meeting
### Conditions handling in Belle2
* Benedikt, or Martin Ritter
### Feedback from FCC tutorial for snowmass
* Common question: What are the different branches in the root file?
* ``uproot`` was also used to access the root file
* Basically re-implemented event store
* Associations
* Try to get this into the, e.g., podio repository
### Next meeting:
* January 12, 2020
* Zoom
There are minutes attached to this event.
Show them.
