EDM4hep Discussion

Tuesday 3 Dec 2024, 09:00 → 10:00 Europe/Zurich

Zoom

Live Notes

EDM4hep Live Notes
==================

Date: Dec 3, 2024
Indico: https://indico.cern.ch/event/1484070/

Connected: Jacopo, Mateusz, Juan, Andre, Thomas, Birgit, Brieuc, Frank, Swathi, Alvaro, Joseph

Apologies: Tao

## Introduction and General Points

### Upcoming workshops / conferences
https://github.com/orgs/key4hep/projects/4/views/1
    
## Progress and discussion
    
## Podio
* https://github.com/AIDASoft/podio/pulls
* https://github.com/AIDASoft/podio/issues
* https://github.com/orgs/AIDASoft/projects/2/views/1

 
### Merged PRs
* Introduce new `links` category for the YAML definitions [#691](https://github.com/AIDASoft/podio/pull/691)
* Make sure that link weights are stored as a struct [#710](https://github.com/AIDASoft/podio/pull/710)
* Add a tool to merge several podio files into a single one [#681](https://github.com/AIDASoft/podio/pull/681)
* CI / pre-commit cleanups:
    * Run `clang-tidy` last for CI [#707](https://github.com/AIDASoft/podio/pull/707)
    * Remove no longer available option from `.clang-tidy` [#709](https://github.com/AIDASoft/podio/pull/709)

### Add a `LinkNavigator` utility
* https://github.com/AIDASoft/podio/pull/646
* Simlar to LCIO `UTIL::LCRelationNavigator`. Facilitates navigating links
* Makes use of templated links to improve usability and reduce potential for confusion

### Allow types from different data models in interfaces
* https://github.com/AIDASoft/podio/pull/714
* Fix for https://github.com/AIDASoft/podio/issues/611
* Introduce a small helper type that type erases the necesary comparison operator and a free function to create it
    * Without leaking the internal `Obj*`!
* Possible after [#673](https://github.com/AIDASoft/podio/pull/673) removed some of the original restrictions
* Performance really an issue?
    * Usage mostly for `std::map` / `std::set`
* Keep `operator==` as is

### [WIP] Store the collection information in a struct instead of a tuple
* https://github.com/AIDASoft/podio/pull/711
* Makes `TTree` version store information as `struct` instead of `std::tuple`
* RNTuple version already stores info without `std::tuple`, **but in different format**
* [ ] Harmonize formats?

### Add support for reading several rntuple files
* https://github.com/AIDASoft/podio/pull/708
* [ ] Waiting for c++20 support to have `std::ranges` available (i.e. [#698](https://github.com/AIDASoft/podio/pull/698))

### Add a simple test that only reads partial frames to check proper cleanup
* https://github.com/AIDASoft/podio/pull/713
* Follow up on issue [#500](https://github.com/AIDASoft/podio/issues/500) and discussion at last weeks Key4hep meeting
* [ ] Check if truly necessary or covered by other tests?

### Improve the detection of schema changes and add tests 
* https://github.com/AIDASoft/podio/pull/715
* Addresses https://github.com/AIDASoft/podio/issues/712
* Make the schema evolution checking script detect changes to `VectorMembers`, `OneToManyRelations` and `OneToOneRelations`
* Add tests to make sure this actually does the right thing
    
### Require c++ 20 and remove c++17 support
* https://github.com/AIDASoft/podio/pull/698
* https://github.com/AIDASoft/podio/pull/700 (make c++17 a warning at cmake stage)
* [x] How formal do we want / need to be with this wrt versioning?
    * Technically a breaking change
    * podio has `SameMajorVersion` as compatibility setting in exported CMake configs -> will break in places where we have `find_package(podio 1.0)`
* **Put warning into next tag. Remove in next-to-next tag and don't make a v2.0** 
* [ ] On hold until v01-02 tag has been done

## EDM4hep
* https://github.com/key4hep/EDM4hep/pulls
* https://github.com/key4hep/EDM4hep/issues
* https://github.com/orgs/key4hep/projects/5 

### Drift chamber digi
* https://github.com/key4hep/EDM4hep/pull/385
* Add new `SensitiveWireHit` that results from drift chamber digitization
    * Before ambiguity resolution (no real 3D position defined)
    * Distance to wire is a circle (defined by time of arrival and drift velocity)
* Added to `TrackerHit` interface
    * Position is not well defined for `SensitiveWireHit` but need `position` for interface
    * `type` and `quality` unused at the moment in `SensitiveWireHit` but probably OK to leave in 
    * Position is the wire that is closest to the hit.
    * `position` could be renamed to something more specific and then `getPosition` can be defined in `ExtraCode`
* Might still need some adjustments
    * Used in digitization but reconstruction will potentially require some changes
* Working in extension becomes cumbersome for low level types
    * Need to redefine things *on top of it* for working with it
    * Essentially creating a parallel data format
    * Might be workable if interfaces work across extensions
* No uncertainties stored yet
    * Attach uncertainties even if they are the same for all hits? -> For agnostic datamodel, yes. 
    * Check other hits for consistency, e.g. can `position` / `direction` be expressed in 2D + 3D position of the wire (from `cellID`)?
    * [ ] Try to find names for uncertainties to see if concepts are clear
* Usable for other "drift chamber like" detectors, e.g. straw tubes
* `TrackerHitPlane` has a `position` in 3D which is redundant (i.e. can be used without `cellID`)
 
### RawHits removed from TrackerHits
* https://github.com/key4hep/EDM4hep/issues/382
* Breaks usage for EIC
* Can probably be brought back as `podio::ObjectID` instead of `edm4hep::ObjectID`
* [ ] Confirm schema evolution is technically possible
* [ ] Decide on timeline for this (after EDM4hep v01-00)
* [ ] Can we use `links` for this?

### [WIP] Switch to templated links and remove explicit declarations from yaml file
* https://github.com/key4hep/EDM4hep/pull/373
* Transparent reading of old (generated) link collections
    * Some "schema evolution" code necessary
* todays nightlies should include the necessary functionality in podio to check this in CI

### Added SimDRCalorimeterHit for dual-readout
* https://github.com/key4hep/EDM4hep/pull/380 
* Crystal dual-readout team, tbc. what the goal is here
* To be seen whether this can be handled by existing CalorimeterHit

    
### Zenodo Batch, Author / Contributor list
* https://github.com/key4hep/EDM4hep/pull/375
* [ ] Prepare a `CITATION.cff` file to point to some publication (once it's ready).
* Decide on a case by case basis on whether inclusion to the authors is warranted

### Run information storage
* Created issue for further: https://github.com/key4hep/EDM4hep/issues/386

    
## Converter & MarlinWrapper

## AoB

* key4hep-spack install script generation goes into infinite loop
    * https://github.com/key4hep/key4hep-spack/issues/678

## Next meeting:
* Dec 17, 9:00 CET

09:00 → 09:05 Introduction

Speakers: Andre Sailer (CERN), Frank-Dieter Gaede (Deutsches Elektronen-Synchrotron (DE)), Thomas Madlener (Deutsches Elektronen-Synchrotron (DESY))

EDM4hep Github

Podio Github

09:10 → 09:30 DriftChamberHit discussion

Speakers: Alvaro Tolosa-Delgado (CERN), Brieuc Francois (CERN)

09:50 → 09:51 Discussion

Speaker: Dr All