CLICdp + Key4hep Software Discussion

Europe/Zurich
ZOOM

ZOOM

Andre Sailer (CERN)

Software Meeting Minutes
---

# 220125: Software Meeting

Agenda: https://indico.cern.ch/event/1120014/

Connected: Andre, Benedikt, Placido, Valentin
Present:

Apologies:

## Proceedings/Talks/etc.

* ECFA Simulation Meeting: Feburary 1
    * https://indico.cern.ch/event/1097819/
    * Andre: Presenting CLIC Full Simulation
    * Valentin: CLD and IDEA Full Simulation

* FOSDEM: February 5-6
    * https://fosdem.org/2022/
    * https://hpc-bigdata-fosdem22.github.io/

* Liverpool FCC meeting: February 7-11
    * https://indico.cern.ch/event/1066234/
    * Andre: Presenting CLD detector
    
* ACAT Proceedings
    * February 13, 2022

## SummerStudent project(s)?

* Continue tracking studies: start using the timing information
* ~~Validation for results (jet energy, tracking resolution), but we should do this before the summer~~
* IO Backends or performance measurements or both (GSOC)
* Converting algorithms from iLCSoft to Gaudi (Placido, Benedikt)
    * Including the validation
* EDM4hep output for GuineaPig (Valentin, Andre)
    * https://gitlab.cern.ch/clic-software/guinea-pig/
    * can store metadata about configuration in the single outputfile
* julia frontend for podio

## MC Generators

## CLICDet Delphes Card

## Simulation

* Issue with simulation of muons with fixed pT?
    * observed different duplicate vs phi using different muon simulation runs
    * Maybe different input (not pt, but energy)?
    * Different Theta range, 0-90 degrees before, 0-180 degrees now

## Reconstruction

### Tracking

### Particle Flow

### Digitizer

### Event Display

### FlavourTagging

### Long-Lived Particle

### dE/dX

### LCGeo Test

## Infrastructure

### Sanitizers/Compilers

## iLCDirac

* FCC VO

* add key4hep-stack to config  (gaudiapp)
    * k4run executable (modifiable)
    * Test Clic Reco With GaudiWrapper inside iLCDirac
        * [x] InputFiles
            * --EventDataSvc
            * --LcioInput
        * [x] OutputFile
            * PodioOutput
        * [x] CompactFile
            * -->GeoSvc (to come to k4Sim)
        * [x] NumberOfEvents
            * -n , --ApplicationMgr.EvtMax
        * [ ] RandomSeed
            * Not fully implemented in vanilla Gaudi
            * Should implement something in Gaudi like the randomseeder in Marlin

* [ ] Next Workflow for FCC
    * [ ] Pythia via Gaudi: need command and steeringfile(s) to use
        * [ ] Check mattermost channel for command and steeringfile
    * [x] Can maybe overwrite settings from Pythia File via the command line options


## Key4hep

### Gaudi

* [ ] Make a service optional/disable in config file: Ticket with Gaudi
    * https://gitlab.cern.ch/gaudi/Gaudi/-/issues/178
* [ ] How to gracefully stop execution "right now"? Re-implement ApplicationMgr?
    * [ ] Reset number of events to be run over?
* [ ] Random Seeding


### CI

* [ ] CVMFS deploy from gitlab: https://cern.service-now.com/service-portal?id=ticket&table=u_request_fulfillment&n=RQF1650416
    * --> Need to re-open the ticket (Valentin)
    * gitlab-runner  was missing permissions
        * New ticket: https://cern.service-now.com/service-portal?id=ticket&table=u_request_fulfillment&n=RQF1747119
        * IT will fix the gitlab integration on another repo and then replay the steps on sw.hsf{-nightlies}.org

* [ ] Create a MacOS11 build for podio
    * [ ] Use LCG mac11 nightly builds (or mac1015)
    * https://github.com/AIDASoft/podio/pull/225
    * [x] Not using podio master in LCG nightly builds?
        * Not using the master, using tag
    * mac11 fuse on github actions is not working
        * [ ] issue with github-actions


### Environment Script

### View

### key4hep-spack/spack

* centos8 build available
    * [ ] to be added to the CI

* spack dev setup, how to?
    * [ ] documentation needs some additions: creating environment, using packages for development

### podio

- [ ]  podio RNTuple backend
    - [ ] Write collections, where only the classname is known as a string is working, reading must also work somehow
    - [x] Metadata,
    - [ ] subset collections
    - [ ] etc

### EDM4hep

* When writing EDM4hep collections Clusters and CaloHits, but only writing the
  ECAL hit collection, not the HCAL hits, which might be linked from the cluster,
  things break when reading the file that only contains the Clusters or only some of the HitCollections
  * [ ] Test and Fixes: https://github.com/AIDASoft/podio/pull/235

* [ ] UserClasses, discuss in podio/edm4hep meeting if direct access to underlying pods is possible or desirable
    * [x] https://github.com/AIDASoft/podio/pull/213
    * [ ] Needs to be merged so that userfloat etc. functionality in  k4FWCore can be replaced by it

* [ ] Test writing sample event data:
    * Incomplete, e.g., TrackCollection not written

* [ ] TrackerHit"Wrapper"


### k4FWCore

- [ ] Using more podio stuff directly for the event service, GaudiHive like interface

### k4MarlinWrapper

* [ ] geoSvc to replace MarlinDD4hep
    * [ ] Need to set the GlobalTrackerReadoutEncodingStringID

* [x] Adding conversions for all metadata https://github.com/key4hep/k4MarlinWrapper/pull/50
* [x] Some exception happening during reconstruction with edm4hep input
    * [x] add test with edm4hep input
* [ ] Add treatment for Marlin Exceptions
* [ ] Multithreading
    * [ ] needs multithreading capable podio/k4FWCore
    * [ ] LCFIPlus: jetClustering and Refiner giving trouble
        * [ ] static variable https://github.com/lcfiplus/LCFIPlus/blob/278d7bb54e2602c2c0cd170c00f72fe302794bcf/include/LcfiplusProcessor.h#L59

* [ ] some issues discovered by Erica in the converters

* [ ] change clic steering python file to use environment variable to locate other steering input files.

### spack

Issue

Opened PRs for a bunch of improvements:
* [ ] fixes to fetching fastjet https://github.com/spack/spack/pull/20064
* [ ] special "commit" version for nightlies https://github.com/spack/spack/pull/20065

* Compiling everything with clang(12)
    * Issue with "base" packages, mesa
    * Requires patches for some packages to existing versions

* Updating to latest spack version
* HSF packaging WG presentation by Todd Gamblin spack author

* DD4hep recipe:
    * Which versions to keep?
        * Keep from what is in the last ILD iLCSoft build
        * /cvmfs/ilc.desy.de/sw/x86_64_gcc82_centos7/v02-02-03/DD4hep/v01-11-02/

## FCCee Detector Model

* Andrea Ciarma investigating issues with the Simulation job options

## SCT Integration

* Directly adopt key4hep (Gaudi/k4fwcore from hsf spack stack) fo Aurora?

## Documentation Update

* Take a look at Benedikt's changes
* https://github.com/key4hep/key4hep-web
    * How does this integrate with https://key4hep.github.io/key4hep-doc/ ?
    * Is there a HTML version of this to better understand what it looks like?


## AOB

### Next Meeting

February 8, 2021, 13:00

 

 

There are minutes attached to this event. Show them.
    • 13:00 14:00
      Round the table 1h