EDM4hep Discussion

Europe/Zurich
Zoom

Zoom

Zoom Meeting ID
98484040528
Host
Andre Sailer
Useful links
Join via phone
Zoom URL

EDM4hep Live Notes
==================

Date: Oct 14, 2025
Indico: https://indico.cern.ch/event/1600615/

Connected: Juraj, Andreas, Andre, Juan, Thomas, Brieuc, Mateusz

Apologies:


### Upcoming workshops / conferences
https://github.com/orgs/key4hep/projects/4/views/1


## Progress and discussion

### Logo Poll
- **Final chance to vote**
- https://github.com/orgs/key4hep/discussions/3


## Podio
* https://github.com/AIDASoft/podio/pulls
* https://github.com/AIDASoft/podio/issues
* https://github.com/orgs/AIDASoft/projects/2/views/1

### New tag v01-05
* https://github.com/AIDASoft/podio/releases/tag/v01-05
* DD4hep v01-32 (and previous) cannot use this
    * Need https://github.com/AIDASoft/DD4hep/pull/1480 
 
### Merged PRs
- Add an indicator for subset collections to podio-dump [#855](https://github.com/AIDASoft/podio/pull/855)
- podio-merge-files: import later for faster help messages and use get_reader [#857](https://github.com/AIDASoft/podio/pull/857)
- Add documentation about some of the existing tools [#854](https://github.com/AIDASoft/podio/pull/854)
- Generate old POD versions into a version namespace [#803](https://github.com/AIDASoft/podio/pull/803)

### `isValid()` for collections does not return what one would think that it means
* https://github.com/AIDASoft/podio/issues/853
* https://github.com/AIDASoft/podio/pull/860
* Deprecate `isValid` and replace by `hasID` (same behavior, more approriate name)
* What to do with podio version once `isValid` is removed?
* Not ussed too much(?)

### `ROOTReader` cannot read a file that only contains subset collections
* https://github.com/AIDASoft/podio/issues/859
* Dictionaries not loaded if only subset collections are in file
* Not sure if there is an easy fix for this that doesn't tank performance
* Files with only subset collections are effectively useless, as there is no data to point to.
* [ ] Add a warning that a file only contains subset collections

### Add the possibility to extract collections from a Frame
* https://github.com/AIDASoft/podio/pull/856
* Feature request from Acts (Paul)
    * Not sure if the actual problem they have is not better solved with an appropriate DD4hep ParticleHandler
    * Aware of the existing functionality in podio to do the remapping (e.g. `clone`-ing collections and manually re-setting relations)?
    * could also use the type lists that we generate to do some variadic template things?
* Might still be useful as an "expert feature"?
    * Could be useful for RDataFrame for mutating a collection(?)
    

### Improve error messages when readers encounter problems
* https://github.com/AIDASoft/podio/pull/851
* https://github.com/AIDASoft/podio/issues/847
* https://github.com/AIDASoft/podio/issues/848
* Give users a bit more information than simply running into uncaught `std::bad_function_call`
* Can we do better for error recovery?
    * For ROOT: we either get the full Frame or nothing, but we could probably do this on a collection basis as well(?)
    * For SIO: we could tackle this on a collection basis (not done in PR yet)
* For `Frame` we already do the permissive way of returning an empty collectoin or a `nullptr` if not found.
    * Do the same for the Readers for consistency

### Schema evolution
* Refactored testing in podio related to schema evolution quite extensively
* Several PRs in flight, need to clean them up and figure out the actually necessary changes
    * https://github.com/AIDASoft/podio/pull/845 (**ready for review**)
    * https://github.com/AIDASoft/podio/pull/828 


### Make sure to not consume all available memory on garbage data
* ~~https://github.com/AIDASoft/podio/pull/842~~ https://github.com/AIDASoft/podio/pull/846
* Discovered in testing, where it can be mitigated [#836](https://github.com/AIDASoft/podio/pull/836)
* [ ] Try to make a reproducer and see if this is a ROOT issue
* [ ] Verify this crashes with debug builds (of podio) of Key4hep stack
* Can we try to store some "parity" information, e.g. the actual collection size to check whether what we read is correct?
    * Might break with `size() == 0`, i.e. empty collections
* Not yet happened in the wild, only occured in testing so far, but once schema evolution is in place people might stumble over this
    * Solve later?
* Can we catch this earlier?
    * E.g. check information from ROOT file to see if things match
* No clear decision yet how to best handle this
* [x] Confirm that RNTuple is not affected
* [ ] Review and merge

    
## EDM4hep
* https://github.com/key4hep/EDM4hep/pulls
* https://github.com/key4hep/EDM4hep/issues
* https://github.com/orgs/key4hep/projects/5 


### Merged PRs
- Use edm4hep.h in edm4hep2json to avoid hardcoding the types [#443](https://github.com/key4hep/EDM4hep/pull/443)

### Do not build and extra library for the old schemas 
* https://github.com/key4hep/EDM4hep/pull/452

### Rework the generator related information structs
* https://github.com/key4hep/EDM4hep/pull/446
* https://github.com/key4hep/EDM4hep/issues/409
* Remove `GeneratorPdfInfo` / partially merge it into `GeneratorEventParameters`
* **Existing files with these datatypes will break**
    * [x] Has anyone used these outside of k4GeneratorsConfig?
    * [ ] Check if [podio#851](https://github.com/AIDASoft/podio/pull/851) can be used here to not break
* [x] Verify that we can indeed store all the necessary information in this class
    * Keep in mind that we also have file and event level parameters to store more things
    * Seems to be a rather minimal set. Adding new things is easier (schema evolution wise than removing /changing existing thigns)
* Not bumping schema version (has already been bumped for MCParticle changes)

## Converter & MarlinWrapper


## AoB
* ROOT release plan: https://indico.cern.ch/event/1590848/contributions/6704085/attachments/3146168/5586196/251001%20-%20ROOT%20Q3%202025%20Report.pdf (S13)


## Next meeting:
* Oct 28, 09:00

There are minutes attached to this event. Show them.
    • 09:00 09:05
      Introduction 5m
      Speakers: Andre Sailer (CERN), Frank-Dieter Gaede (Deutsches Elektronen-Synchrotron (DE)), Thomas Madlener (Deutsches Elektronen-Synchrotron (DESY))
    • 09:50 09:51
      Discussion 1m
      Speaker: Dr All