EDM4hep Discussion

→ Europe/Zurich
Zoom

Zoom

Zoom Meeting ID
98484040528
Host
Andre Sailer
Useful links
Join via phone
Zoom URL

EDM4hep Live Notes
==================

Date: September 24, 2024
Indico: https://indico.cern.ch/event/1459928/

Connected: Juan, Andre, Tao, Swathi, Mateusz, Juraj, Jacopo, Leonhard, Sungwon, Thomas, Brieuc, Pere, Benedikt, Frank

Apologies:

## Introduction and General Points

### Upcoming workshops / conferences
https://github.com/orgs/key4hep/projects/4/views/1
    
## Progress and discussion
    
## Podio
* https://github.com/AIDASoft/podio/pulls
* https://github.com/AIDASoft/podio/issues
* https://github.com/orgs/AIDASoft/projects/2/views/1

### New tag `v01-01`
* [release on github](https://github.com/AIDASoft/podio/releases/tag/v01-01)
 
### Merged PRs
* Make `CreateDataFrame` available from python ([#674](https://github.com/AIDASoft/podio/pull/674))
* Switch to `static_assert` for better compiler errors related to unsupported parameter types ([#676](https://github.com/AIDASoft/podio/pull/676), [#680](https://github.com/AIDASoft/podio/pull/680))
* Add a version module that allows to compare versions in python ([#678](https://github.com/AIDASoft/podio/pull/678))
* Remove deprecated readers and writers ([#684](https://github.com/AIDASoft/podio/pull/684))
* Documentation, CI and ohther minor fixes
    * Fix a few wrong `@param`s ([#675](https://github.com/AIDASoft/podio/pull/675))
    * Build podio with all available features for EDM4hep CI workflows [#677](https://github.com/AIDASoft/podio/pull/677) 
    * Fix printout when data source is not available ([#679](https://github.com/AIDASoft/podio/pull/679)
    * DataSource: implement missing GetLabel method ([#682](https://github.com/AIDASoft/podio/pull/682))
 
    
### Templated links between arbitrary datatypes
* [ ] https://github.com/AIDASoft/podio/pull/257
* Ready for (final) review.
* Some fixes / features required to make switch in EDM4hep transparent
    * E.g. support for interface types in links
    * Details of python binding overload resolution involving implicit conversions

### Rework the I/O handling for relations ([#673](https://github.com/AIDASoft/podio/pull/673))
* Necessary for links with interface types
* Generic solution that "simply" moves some of the special casing from jinja2 templates to proper c++ templates

### Add a tool to merge several podio files into a single one ([#681](https://github.com/AIDASoft/podio/pull/681))
* [ ] Add metadata about which files were merged

### Use `enable_if` ot allow using containters of interfaces with object wrappers ([#683](https://github.com/AIDASoft/podio/pull/681))
* Originally triggered by [k4FWCore#238](https://github.com/key4hep/k4FWCore/issues/238)
    * In principle we want to have concrete collection types for passing data around
    * `std::vector<TrackerHit>` should be a special case. Could check in k4FWCore to allow for interface types
    * Check if interface types can have collections?


## EDM4hep
* https://github.com/key4hep/EDM4hep/pulls
* https://github.com/key4hep/EDM4hep/issues
* https://github.com/orgs/key4hep/projects/5 

### New tag `v00-99-01`
* [release on github](https://github.com/key4hep/k4FWCore/issues/238)

### Merged PRs

* Make sure that files with the complete datamodel can be read ([#361](https://github.com/key4hep/EDM4hep/pull/361), [#368](https://github.com/key4hep/EDM4hep/pull/368), [#369](https://github.com/key4hep/EDM4hep/pull/369))
* Add loading pythonizations from podio ([#290](https://github.com/key4hep/EDM4hep/pull/290))
* Inject the EDM4hep version into the metatdata that are stored by podio ([#362](https://github.com/key4hep/EDM4hep/pull/362))
* Bump the minimum podio version to have all necessary features ([#374](https://github.com/key4hep/EDM4hep/pull/374))
* Fix the HepPDT related cmake warning and make sure that all tests build ([#367](https://github.com/key4hep/EDM4hep/pull/367), [#371](https://github.com/key4hep/EDM4hep/pull/371), [#372](https://github.com/key4hep/EDM4hep/pull/372))
* Minor fixes / changes
    * Use the Key4hepConfig flag to set the standard, compiler flags and rpath magic ([#359](https://github.com/key4hep/EDM4hep/pull/359))
    * Remove comments in edm4hep.yaml that don't add any information ([#366](https://github.com/key4hep/EDM4hep/pull/366))
    * Add missing attributes for python package ([#363](https://github.com/key4hep/EDM4hep/pull/363))

    
### Add read tests to ensure backwards compatibility
* https://github.com/key4hep/EDM4hep/pull/358
* Similar to what we do in podio to ensure we can truly read files written with older versions
* Pre-requisite for switching out links with templated links
* Test files for EDM4hep v00-99-01 + podio v01-01

### add formatting and linting python in pre-commit 
* https://github.com/key4hep/EDM4hep/pull/370 
* Discussion about which parts of the configuration of different tools should be centralized and which parts should be package specific
    * `ruff` configuration can be layered, i.e. can have different config files with different priority
    * Major issue for linting: Gaudi is dynamically loaded
    * Make formatting centralized and apply to all repos
* [ ] Need to have a dedicated discussion for python style (e.g. type annotations, etc.)

    
### [WIP] Switch to templated links and remove explicit declarations from yaml file
* https://github.com/key4hep/EDM4hep/pull/373
* Transparent reading of old (generated) link collections
    * Some "schema evolution" code necessary
* Example of how adding a new link looks like at the moment
    * Create a header with typedef
    * Use `PODIO_DECLARE_LINK` to instantiate the necessary buffer creation code for a given type
* Make a new dedicated `links`(?) category in the YAML file?
    * Yes, keeps EDM4hep self-contained
    * [ ] Make a separate PR that introduces that
* Do we need schema evolution from non-templated links? 
    * Yes, we had a tagged version (`v00-99`) of EDM4hep with the non-templated links
    
    
## Converter & MarlinWrapper

* `IOSvc` support for MarlinWrapper?
    * https://github.com/key4hep/k4MarlinWrapper/issues/202
    * Do we need it? `IOSvc` is mainly for multithreading, but most processors are not threadsafe
    * [ ] Can we change the segmentatoin fault to an error message?

## AoB

* Pick up putting repositories on zenodo

## Next meeting:
* Oct 08, 2024, 09:00

There are minutes attached to this event. Show them.
    • 09:00 → 09:05
      Introduction 5m
      Speakers: Andre Sailer (CERN), Frank-Dieter Gaede (Deutsches Elektronen-Synchrotron (DE)), Thomas Madlener (Deutsches Elektronen-Synchrotron (DESY))
    • 09:50 → 09:51
      Discussion 1m
      Speaker: Dr All