EDM4hep Discussion

Tuesday 30 Sept 2025, 09:00 → 10:00 Europe/Zurich

Zoom

Live Notes

EDM4hep Live Notes
==================

Date: Sep 30, 2025
Indico: https://indico.cern.ch/event/1591340/

Connected: Fangyi, Brieuc, Andreas, Juraj, Tao, Joseph, Juan, Thomas, Frank

Apologies:

### Upcoming workshops / conferences
https://github.com/orgs/key4hep/projects/4/views/1

## HGCCAL Beam test data processing in Key4hep
* See presentation
* S13: Ideal solution would be that DAQ effectively write podio generated EDM data directly.
    * Does obviously not work for already finished testbeams
    * Does not necessarily have to be EDM4hep
* S14: MC digitization could output Raw data instead of "already" digitized hits
    * Current digitizers in Key4hep almost always do an energy smearing and go to "calibrated" data directly
    * Splitting into several steps might lose link between SimHit and RawCalorimeterHit
    * Two step calibration exists in principle for ILD, but doesn't create RawCalorimeterHit in between steps 
* RawCalorimeterHit not yet used by real test beal. CALICE used their own raw data format
    * For this specific case one could fit two 16 bits into the existing ADC count
    * Dumping everything (for all testbeams) into a RawCalorimeterHit might become unattainable, because it just will have potentially a lot of unused members if several appraoches are merged
    * We will likely not be able to handle all test beam use cases with one type
    * Could try to define commonly used members in RawCalorimeterHit and try to extend it with special hits for different test beams and use them via Links
    * **Open for suggestions via PRs**
* DD4hep does not require a 1:1 mapping from Volume to CellID. Can use segmentation to have different CellIDs for the same volume.
    * Done for pretty much all highly granular calorimeters
    * Would require to write a dedicated readout action to do
    * Can foresee additional (unused in simulation) bits for simulation
* Goal would be to have as much usage for Key4hep in test beams as quickly as possible
    * Raw data will often be messy. Will probably have to have some coherent processing attached to specific test beams.
    * Potential intermediate goal: Testbeams produce calibrated `CalorimeterHit`s for further analysis

## Progress and discussion

### Logo Poll
- **Final chance to vote**
- https://github.com/orgs/key4hep/discussions/3

## Podio
* https://github.com/AIDASoft/podio/pulls
* https://github.com/AIDASoft/podio/issues
* https://github.com/orgs/AIDASoft/projects/2/views/1
 
### Merged PRs
- Make sure SioBlock libraries are properly loaded from PODIO_SIOBLOCK_PATH [#852](https://github.com/AIDASoft/podio/pull/852)
- Refactor schema evolution detection [#844](https://github.com/AIDASoft/podio/pull/844)
- Remove support for CPack [#849](https://github.com/AIDASoft/podio/pull/849)
- Set ROOT_LIBRARY_PATH in tests to better control the environment [#841](https://github.com/AIDASoft/podio/pull/841)

### Schema evolution
* Refactored testing in podio related to schema evolution quite extensively
* Several PRs in flight, need to clean them up and figure out the actually necessary changes
    * https://github.com/AIDASoft/podio/pull/803 (**ready for review**)
    * https://github.com/AIDASoft/podio/pull/845 (**ready for review**)
    * https://github.com/AIDASoft/podio/pull/828 
* **Plan: merge [#803](https://github.com/AIDASoft/podio/pull/803) and make a tag**

### Improve error messages when readers encounter problems
* https://github.com/AIDASoft/podio/pull/851
* https://github.com/AIDASoft/podio/issues/847
* https://github.com/AIDASoft/podio/issues/848
* Give users a bit more information than simply running into uncaught `std::bad_function_call`
* Can we do better for error recovery?
    * For ROOT: we either get the full Frame or nothing
    * For SIO: we could tackle this on a collection basis (not done in PR yet)

### Make sure to not consume all available memory on garbage data
* ~~https://github.com/AIDASoft/podio/pull/842~~ https://github.com/AIDASoft/podio/pull/846
* Discovered in testing, where it can be mitigated [#836](https://github.com/AIDASoft/podio/pull/836)
* [ ] Try to make a reproducer and see if this is a ROOT issue
* [ ] Verify this crashes with debug builds (of podio) of Key4hep stack
* Can we try to store some "parity" information, e.g. the actual collection size to check whether what we read is correct?
    * Might break with `size() == 0`, i.e. empty collections
* Not yet happened in the wild, only occured in testing so far, but once schema evolution is in place people might stumble over this
    * Solve later?
* Can we catch this earlier?
    * E.g. check information from ROOT file to see if things match
* No clear decision yet how to best handle this
* [ ] Confirm that RNTuple is not affected
    
 ## EDM4hep
* https://github.com/key4hep/EDM4hep/pulls
* https://github.com/key4hep/EDM4hep/issues
* https://github.com/orgs/key4hep/projects/5 

### Merged PRs
- Fix missing arcticle data in citation file [#450](https://github.com/key4hep/EDM4hep/pull/450)
- Add documentation about the subtle differences wrt LCIO [#447](https://github.com/key4hep/EDM4hep/pull/447)
- Set ROOT_LIBRARY_PATH for tests [#451](https://github.com/key4hep/EDM4hep/pull/451)
- remove mentions of removed un-namespaced labels [#448](https://github.com/key4hep/EDM4hep/pull/448)
- Remove support for CPack and the unused `EDM4HEP.pc.in` [#445](https://github.com/key4hep/EDM4hep/pull/445)
- Fix another bunch of typos in docstrings and comments [#449](https://github.com/key4hep/EDM4hep/pull/449)

### Rework the generator related information structs
* https://github.com/key4hep/EDM4hep/pull/446
* https://github.com/key4hep/EDM4hep/issues/409
* Remove `GeneratorPdfInfo` / partially merge it into `GeneratorEventParameters`
* **Existing files with these datatypes will break**
    * [ ] Has anyone used these outside of k4GeneratorsConfig?
    * [ ] Check if [podio#851](https://github.com/AIDASoft/podio/pull/851) can be used here to not break
* [ ] Verify that we can indeed store all the necessary information int his class
    * Keep in mind that we also have file and event level parameters to store more things

### Use edm4hep.h in edm4hep2json to avoid hardcoding the types
* https://github.com/key4hep/EDM4hep/pull/443
* [ ] New podio tag

## Converter & MarlinWrapper

## AoB

## Next meeting:
* Oct 14, 09:00

09:00 → 09:05 Introduction

Speakers: Andre Sailer (CERN), Frank-Dieter Gaede (Deutsches Elektronen-Synchrotron (DE)), Thomas Madlener (Deutsches Elektronen-Synchrotron (DESY))

EDM4hep Github

Podio Github

09:10 → 09:30 HGCCAL Beam test data processing in Key4hep

Speaker: Fangyi Guo (Chinese Academy of Sciences (CN))

TBinCEPCSW.pdf

09:50 → 09:51 Discussion

Speaker: Dr All