EDM4hep Discussion

Tuesday 27 Feb 2024, 09:00 → 10:00 Europe/Zurich

Zoom

Live Notes

EDM4hep Live Notes
==================

Date: Feb 27, 2024
Indico: https://indico.cern.ch/event/1388088/

Connected: Juraj, Brieuc, Frank, Mateusz, Swathi, Pere, Graeme, Leonhard, Juan, Benedikt, Joseph

Apologies: Tao

## Introduction and General Points

### Upcoming workshops / conferences
https://github.com/orgs/key4hep/projects/4/views/1
    
## Progress and discussion

## Virtual sprint for (towards) EDM4hep 1.0
- **Tagged all involved packages before to be able to make a release with them**
- Remove edm4hep::ObjectID: https://github.com/key4hep/EDM4hep/pull/260
    - Also removed them as `VectorMembers` from `TrackerHit3D` and `TrackerHitPlane`
- Rename `MCParticle` relation to `particle`: https://github.com/key4hep/EDM4hep/pull/256
    - Transparent via `ExtraCode` + deprecation warning
- Vector of weights in `EventHeader`: https://github.com/key4hep/EDM4hep/pull/254
- Introdue `TrackerHit` interface and rename existing `TrackerHit` to `TrackerHit3D`: https://github.com/key4hep/EDM4hep/pull/252
    - Make the converter attach converted `TrackerHitPlane` and `TrackerHit3D` to `Track`: https://github.com/key4hep/k4EDM4hep2LcioConv/pull/49
    - Quite a few related PRs to make this change transparent so that we had some sort of CI in the meantime (see PR above)
- MCParticle momenta as doubles: https://github.com/key4hep/EDM4hep/pull/267
- Rename `type` to `PDG` in `ReconstructedParticle`: https://github.com/key4hep/EDM4hep/pull/273
    - (Almost) Transparent via `ExtraCode` + deprecation warning
- Make capitalization of `eDep` consistent throughout datamodel: https://github.com/key4hep/EDM4hep/pull/274
- Remove (unused) `BITEndpoint` from `MCparticle`: https://github.com/key4hep/EDM4hep/pull/275
- Fix docstrings and homogenize authors: https://github.com/key4hep/EDM4hep/pull/276
- Use new style for unit declarations: https://github.com/key4hep/EDM4hep/pull/277
- Introduce `operator!=` for `VectorNx` components via `ExtraCode`: https://github.com/key4hep/EDM4hep/pull/281

#### Unforeseen breakages
- `type` to `PDG` in `ReconstructedParticle`
    - FCCAnalyses: https://github.com/HEP-FCC/FCCAnalyses/pull/354
    - No backwards compatibility yet, more of a quick fix to get nightlies building again.
    
#### Things still left to fix
- CEPCSW has been taken out of the nightlies since we were aware of quite a few things that break and that we couldn't fix in the two days we were working on the rest
    - https://github.com/cepc/CEPCSW/issues/263
    - Try to get this back into nightlies as quickly as possible
- podio schema evolution script does not detect that `Vector3f` -> `Vector3d` is not supported
    - https://github.com/AIDASoft/podio/issues/563
- Cleanup to switch to non-deprecated accessor methods:
    - https://github.com/key4hep/k4SimDelphes/pull/118
    - https://github.com/key4hep/k4EDM4hep2LcioConv/pull/55
    
### More things (re)discovered during sprint
- Usage differences of `float` vs `double` wrt LCIO: https://github.com/key4hep/EDM4hep/issues/265
- Subtle differences in integer type vs LCIO: https://github.com/key4hep/EDM4hep/issues/271
- Jet parameters: https://github.com/key4hep/EDM4hep/issues/192
    - Discovered nothing actionable during sprint
- Dedicated covariance matrix components (ala EDM4eic): https://github.com/key4hep/EDM4hep/issues/154 and https://github.com/key4hep/EDM4hep/issues/262
    - [ ] Give it a try
- Time field in `SimCalorimeterHit`: https://github.com/key4hep/EDM4hep/issues/147
    - [ ] No plans to add this. Close
    
## Data translation in EDM4hep from pre 1.0 to 1.0
- See presentation
- Only has to be done for **one** release before 1.0 and **one** release afterwards
- Possible in one step?
    - E.g. read leaves and recompose in new format
    - Could generate "introspection" functionality
        - Yes, but will be a lot of work and somewhat backend dependent
    - Two step process is backend indpendent

## Podio
* https://github.com/AIDASoft/podio/pulls
* https://github.com/AIDASoft/podio/issues
* https://github.com/orgs/AIDASoft/projects/2/views/1

### Recently merged PRs
- Generate mutable reference getters for Mutable types: https://github.com/AIDASoft/podio/pull/553
- Improved error messages for opening bad files: https://github.com/AIDASoft/podio/pull/559
- Generate `operator!=` for datatypes: https://github.com/AIDASoft/podio/pull/560
- CI updates: https://github.com/AIDASoft/podio/pull/562
- Replace `XYZ MutableXYZ::operator()` with `XYZ(MutableXYZ)` constructor for improved discoverability from cppyy: https://github.com/AIDASoft/podio/pull/564

### Open PRs for v1.0
- Make `podio-vis` deal with upstream EDMS: https://github.com/AIDASoft/podio/pull/561
    - Effectively waiting for Key4hep stack to pick up `py-graphviz` as dependency
- RNTuple API update integration: https://github.com/AIDASoft/podio/pull/547
    - Stabilized again in ROOT, now can proceed to fix it on our end 

### CollectionID value -2
* Redundant? 
* [ ] Need to check what this affects

## EDM4hep
* https://github.com/key4hep/EDM4hep/pulls
* https://github.com/key4hep/EDM4hep/issues
* https://github.com/orgs/key4hep/projects/5 

### Reversal of direction of some relations
- https://github.com/key4hep/EDM4hep/pull/268
- `ParticleID` should point to `ReconstructedParticle`
- Remove `ParticleID` from `Cluster` (unused in ILD standard reconstruction)
    - [ ] Remove in LCIO as well?
    - This is a sort of almost requirement for the reversal, as otherwise the `ParticleID` needs two relations, one to the `Cluster` and one to the `ReconstructedParticle`
    - Brieuc reported a use case for the `Cluster`
        - For now solved with using the `type` member and the `shapeParameters`
- Related PRs:
    - Make the converter deal with this: https://github.com/key4hep/k4EDM4hep2LcioConv/pull/51
    - Fix usage in k4SimDelphes: https://github.com/key4hep/k4SimDelphes/pull/117

## Converter & MarlinWrapper

- Enable compiler warnings and fix them: https://github.com/key4hep/k4EDM4hep2LcioConv/pull/53
    - Switch to non-deprecated overloads: https://github.com/key4hep/k4MarlinWrapper/pull/174
- Refactor EDM4hep -> LCIO conversion for more consistency between conversion directions: https://github.com/key4hep/k4EDM4hep2LcioConv/pull/54
    - Switch to new functionality:
    - https://github.com/key4hep/k4MarlinWrapper/pull/176
    - ParticleID handling builds on top of this

## AoB

## Next meeting:
* Mar 12, 09:00

09:00 → 09:05 Introduction

Speakers: Andre Sailer (CERN), Frank-Dieter Gaede (Deutsches Elektronen-Synchrotron (DE)), Thomas Madlener (Deutsches Elektronen-Synchrotron (DESY))

EDM4hep Github

Podio Github

09:05 → 09:20 Data translation in EDM4hep from pre 1.0 to 1.0

Speaker: Juan Miguel Carceller (CERN)

20240227-carceller-converter.pdf

09:20 → 09:35 ParticleID handling update (postponed)

Speaker: Thomas Madlener (Deutsches Elektronen-Synchrotron (DESY))

pid_new_edm4hep.pdf

09:50 → 09:51 Discussion

Speaker: Dr All