EDM4hep Discussion

Tuesday 17 Dec 2024, 09:00 → 10:00 Europe/Zurich

Zoom

Live Notes

EDM4hep Live Notes
==================

Date: Dec 17, 2024
Indico: https://indico.cern.ch/event/1490463/

Connected: Alvaro, Pere, Mateusz, Brieuc, Swathi, Andre, Tao, Juan, Thomas, Birgit

Apologies:

## Introduction and General Points

### Upcoming workshops / conferences
https://github.com/orgs/key4hep/projects/4/views/1
    
## Progress and discussion

## Key4hep in-person meeting @CERN wrapup
* https://indico.cern.ch/event/1482625/
* https://codimd.web.cern.ch/Qs3quDjTQVWf2Spi6Z_NuA#
* Moving `weights` from `EventHeader` to `GenEventHeader` will break for FCC-hh
* Try to propagate latest version to track finding for drift chamber to cross check usability there
    * `u` and `v` for sensitive wire hit as well? Slightly different meaning in drift chamber. For now keep explicit names of angles.
    
## Podio
* https://github.com/AIDASoft/podio/pulls
* https://github.com/AIDASoft/podio/issues
* https://github.com/orgs/AIDASoft/projects/2/views/1

 
### Merged PRs
* Add a `LinkNavigator` utility [#646](https://github.com/AIDASoft/podio/pull/646)
* Allow types from different data models in interfaces [#714](https://github.com/AIDASoft/podio/pull/714)
* Mark the collection destructor with `override` and remove `#include <array>`[#716](https://github.com/AIDASoft/podio/pull/716)

### Improve the detection of schema changes and add tests 
* https://github.com/AIDASoft/podio/pull/715
* Addresses https://github.com/AIDASoft/podio/issues/712
* Make the schema evolution checking script detect changes to `VectorMembers`, `OneToManyRelations` and `OneToOneRelations`
* Add tests to make sure this actually does the right thing
    * Not the most intuitive implementation, but the whole thing needs a refactoring in any case

### Make collection iterators fulfill *LegacyInputIterator* and `input_iterator`
* https://github.com/AIDASoft/podio/pull/626
* Enables algorithms that require *input iterator* (e.g. `std::find`, `std::min`, `std::max`, ...). Now also possible with the ranges versions!
* [ ] Review
* Up to `random_access_iterator` concept is possible, for legacy only input iterator is possible
* Output iterator not compatible with design of podio

### Delete all non-virtual default destructors
* https://github.com/AIDASoft/podio/pull/717
* *Rule of five* vs *rule of zero*

### [WIP] Store the collection information in a struct instead of a tuple
* https://github.com/AIDASoft/podio/pull/711
* Makes `TTree` version store information as `struct` instead of `std::tuple`
* RNTuple version already stores info without `std::tuple`, **but in different format**
* [ ] Harmonize formats?

### Add support for reading several rntuple files
* https://github.com/AIDASoft/podio/pull/708
* [ ] Waiting for c++20 support to have `std::ranges` available (i.e. [#698](https://github.com/AIDASoft/podio/pull/698))

### Add a simple test that only reads partial frames to check proper cleanup
* https://github.com/AIDASoft/podio/pull/713
* Follow up on issue [#500](https://github.com/AIDASoft/podio/issues/500) and discussion at last weeks Key4hep meeting
* [ ] Check if truly necessary or covered by other tests?

    
### Require c++ 20 and remove c++17 support
* https://github.com/AIDASoft/podio/pull/698
* https://github.com/AIDASoft/podio/pull/700 (make c++17 a warning at cmake stage)
* [x] How formal do we want / need to be with this wrt versioning?
    * Technically a breaking change
    * podio has `SameMajorVersion` as compatibility setting in exported CMake configs -> will break in places where we have `find_package(podio 1.0)`
* **Put warning into next tag. Remove in next-to-next tag and don't make a v2.0** 
* [ ] On hold until v01-02 tag has been done

## EDM4hep
* https://github.com/key4hep/EDM4hep/pulls
* https://github.com/key4hep/EDM4hep/issues
* https://github.com/orgs/key4hep/projects/5 

### Add `edm4hep::Tensor` type for us in ML training and inference
* https://github.com/key4hep/EDM4hep/pull/388
* Via EIC colleagues
* Do we need metadata attached to this somehow? How to know where things are in this tensor?
    *  E.g. shape parameters for Clusters
* Very (too?) generic
* What is the use case for EIC currently?
* Support for pytorch?
* "Waste" of unused data type should be rather small as vector will just be empty
    * Cannot do a `union` in podio generated datamodels

### Drift chamber digi
* https://github.com/key4hep/EDM4hep/pull/385
* Add new `SensitiveWireHit` that results from drift chamber digitization
    * Before ambiguity resolution (no real 3D position defined)
    * Distance to wire is a circle (defined by time of arrival and drift velocity)
* Added to `TrackerHit` interface
    * Position is not well defined for `SensitiveWireHit` but need `position` for interface
    * `type` and `quality` unused at the moment in `SensitiveWireHit` but probably OK to leave in 
    * Position is the wire that is closest to the hit.
    * `position` could be renamed to something more specific and then `getPosition` can be defined in `ExtraCode`
* Might still need some adjustments
    * Used in digitization but reconstruction will potentially require some changes
* Working in extension becomes cumbersome for low level types
    * Need to redefine things *on top of it* for working with it
    * Essentially creating a parallel data format
    * Might be workable if interfaces work across extensions
* No uncertainties stored yet
    * Attach uncertainties even if they are the same for all hits? -> For agnostic datamodel, yes. 
    * Check other hits for consistency, e.g. can `position` / `direction` be expressed in 2D + 3D position of the wire (from `cellID`)?
    * [ ] Try to find names for uncertainties to see if concepts are clear
* Usable for other "drift chamber like" detectors, e.g. straw tubes
* `TrackerHitPlane` has a `position` in 3D which is redundant (i.e. can be used without `cellID`)
 
### RawHits removed from TrackerHits
* https://github.com/key4hep/EDM4hep/issues/382
* Breaks usage for EIC
* Can probably be brought back as `podio::ObjectID` instead of `edm4hep::ObjectID`
* [ ] Confirm schema evolution is technically possible
* [ ] Decide on timeline for this (after EDM4hep v01-00)
* [ ] Can we use `links` for this?

### Switch to templated links and remove explicit declarations from yaml file
* https://github.com/key4hep/EDM4hep/pull/373
* Transparent reading of old (generated) link collections
    * Some "schema evolution" code necessary
* todays nightlies should include the necessary functionality in podio to check this in CI
* [ ] Name of shared library that contains schema evolution bits?
    * `edm4hep_v2` might be confusing
    * `edm4hepOLD`? Need to check whether we can contain all of these parts in one library in the end. **Go for this unless someone complains**
    * Should we care at the moment?
    
### Added SimDRCalorimeterHit for dual-readout
* https://github.com/key4hep/EDM4hep/pull/380 
* Cannot use `SimCalorimeterHit` because crystals create two different signals **in the same volume**
* Clarify whether time is needed in CalorimeterHit or whether it can be taken from the contribution
* Do we need an interface that unifies this with `SimCalorimeterHit`?
    * No immediate use case
* Does the digitization use the differences between scintillation and cerenkov photons?
    * Otherwise we will just duplicate information uselessly
* Need to understand digitization in order for better judging this
    * Have Lorenzo in the loop to compare how fiber based DR did this
    
### Zenodo Batch, Author / Contributor list
* https://github.com/key4hep/EDM4hep/pull/375
* [ ] Prepare a `CITATION.cff` file to point to some publication (once it's ready).
* Decide on a case by case basis on whether inclusion to the authors is warranted

### Run information storage
* Created issue for further: https://github.com/key4hep/EDM4hep/issues/386

    
## Converter & MarlinWrapper

## AoB

* DD4hep EDM4hepReader under developments
    * https://github.com/AIDASoft/DD4hep/pull/1371

* key4hep-spack install script generation goes into infinite loop
    * https://github.com/key4hep/key4hep-spack/issues/678
* NTuplizer
    * What goes into this (to be created) repository? - Not yet clear.
    * [ ] Does Gaudi have some usable service that makes this a bit easier?

## Next meeting:
* Jan 28, 9:00 CET (conflict with FCC physics workshop on Jan 14!)

09:00 → 09:05 Introduction

Speakers: Andre Sailer (CERN), Frank-Dieter Gaede (Deutsches Elektronen-Synchrotron (DE)), Thomas Madlener (Deutsches Elektronen-Synchrotron (DESY))

EDM4hep Github

Podio Github

09:50 → 09:51 Discussion

Speaker: Dr All