EDM4hep Discussion
Vidyo
EDM4hep Live Notes
==================
Date: October 13, 2020
Indico: https://indico.cern.ch/event/965132/
This is a document for taking notes during EDM4hep meetings.
Connected: Clement, Placido, Sang Hyun, Thomas, Weidong, Jiaheng, Valentin, Frank, Wenxing, Tao, Brieuc, Gerri
Apologies: Benedikt
## Introduction and General Points
## Progress and Discussion
### HSF podio Presentation
* Thomas preparing abstract:
* Not too many changes since CHEP?
* Mostly changes under the hood
* Changes: Metadata, edm4hep output, delphes output, ...
### Spack installation
* Updated CVMFS installation:
* /cvmfs/sw.hsf.org/key4hep/setup.sh
* Also used for fcc tutorial
* Can this repository be part of the defaults?
* bundle package: use "rolling release model" and tag by date, or pin versions
* Added more python packages to key4hep-stack
* Spack as a developer tool:
* PR:
* Nightly build running
* Add "last commit" as version to re-build only if changes present
* TODO: publish on CVMFS, need way to copy new versions and create a "setup.sh" script
*
### Nightly Builds and CI
* Entire build CI set up by Joseph
* docker image: key4hep-container on dockerhub
* http://dev1.bitquant.com.hk:8011/
* add tests?
* Now possible to build with commits as versions
* Also built for CVMFS
* https://gitlab.cern.ch/key4hep/k4-deploy/
* mirror of k4-spack in order to use CERN's gitlab/openstack infrastructure
* (ask Valentin to get access)
* EDM4hep CI
* building podio on the fly, but ddsim is not picking up the new library
* try to reproduce locally and see which wrong library is picked up where
## Podio
### PRs
* https://github.com/AIDASoft/podio/pulls
* Fix meta data handling
* https://github.com/AIDASoft/podio/pull/133
* Should go before the SIO PR, this PR refactors the tests
* Merged
### Using SIO as backend
* Frank and Thomas looking into it
* https://github.com/AIDASoft/podio/pull/130
* IO and tests working
* Small differences in behaviour of ROOTReader/Writer and SIOReader/Writer
* Split test datamodel into small libraries separating SIO/ROOT/Datamodel?
* How to load podioSioIO on demand?
* standalone GaudiPluginManager
* https://github.com/hegner/PM4hep
* Add option to make RootIO optional on build/cmake time
* Cmake: better isolation, reduce number of IFs
* CMake functions to created libraries for different backends
* Exercised with EDM4hep, make a PR with the changes
* podio is backward compatible
* Ready for review
### issue w/ ROOT and (vectors of) non-copyable collections
* happens in ROOT 6.22
* PM: there is a patch available in LCG repository
* ROOT team is working on a general solution
### "event class" in podio
* Currently being perceived
### Multi-Threading
* More than one event "in flight" should be possible
* TM: root reader seems to be capable, root writer maybe not
* FG: easier to handle with "event class"
*
### Meta Data
#### Usage of "metadata" for user defined data
* need to check if current implementation addresses all use cases
* need test use-cases
### Issues following MetaData Developments
* cannot write out event data previously read from file
* Issue: https://github.com/AIDASoft/podio/issues/103
* Test: https://github.com/AIDASoft/podio/pull/102
* TM: have fix, but break framework core handling of collections
* there seems to be a fix in the Gaudi ROOTWriter in FCC-EDM !?
* historical reason, fixes and developments should be back ported to PODIO
* ...
#### Writing second file with another tree
* https://github.com/key4hep/K4FWCore/issues/10
* this problem only happens in the GAUDI framework
* if user tries to write an additional ROOT file
* to be addressed
### EventStore
### Schema Evolution
- Version for object descriptions, etc.
- Open issue: https://github.com/AIDASoft/podio/issues/86
### Features
* Subset collections?
## LCIOConverters
* Conversion from LCIO to EDM4HEP is almost completed, but we need more testing for the three associations:
* MCRecoParticleAssociation
* MCRecoCaloAssociation
* MCRecoTrackerAssociation
* LCIOInput, an algorithm wrapper in Gaudi/K4FWCore
* https://github.com/ihep-sft-group/LCIOInput
* K4LCIOReader:
* https://github.com/ihep-sft-group/K4LCIOReader
* Used to reconstruct clusters with Pandora
* Publish Pandora Interface on github
* working on spack recipe for CEPC software. Using `spack install cepcsw` to install CEPCSW.
* https://github.com/key4hep/k4-spack/pull/73
## EDM4hep
https://github.com/key4hep/EDM4hep/pulls
### Tracker Hit
* Q: Tracker hit input to tracking algorithms?
* A: In LCIO different traker hits: Planar and Cylindrical using inheritance. Still needs to be adressed how to do this in EDM4hep.
* Q: Is inheritance needed?
* Q: What to use for Driftchambers?
* Open issue:
### Need review of EDM4hepDelphes output
* Output is not stored if no tracks or towers created (?)
* Allow also other types to be part of the list
* Allow construction in a more general way (VV)
* Need to assume there is a complete list of reconstructed particles?
* Relations:
* FCCSW: Issue with more than one RecoParticle pointing to one MCParticle
* In LCIO to Delphes, conversion happens later than in edm4hep, maybe delphes does de-duplication?
* TODO: open issue in edm4hep with reproducer (VV)
### PRs
* Fix delphes EDM4Hep plugin
* https://github.com/key4hep/EDM4hep/pull/79
* Merged
* Add a mention of the example to the key4hep docs:
* https://github.com/key4hep/key4hep-doc
### Release 1.0
* Need:
* ~~Plugin~~
* ~~Eventheader~~
* ~~Meta Data (Event / Run Parameters)~~
## AOB
### Dual Read-out calorimeter for FCC
* Special data structure used for that simulation
* Tried to use edm4hep, some issues
* Present in a future meeting
### Conditions handling in Belle2
* Benedikt, or Martin Ritter
### Feedback from FCC tutorial for snowmass
* Common question: What are the different branches in the root file?
* ``uproot`` was also used to access the root file
* Basically re-implemented event store
* Associations
* Try to get this into the, e.g., podio repository
### Next meeting:
* October 27, 2020
* NOTE: European Summer Time ends Sunday, October 25!