Speakers
Description
We evaluate two Generative Adversarial Network (GAN) models developed by the COherent Muon to Electron Transition (COMET) collaboration to generate sequences of particle hits in a Cylindrical Drift Chamber (CDC). The models are first evaluated by measuring the similarity between distributions of particle-level, physical features. We then measure the Effectively Unbiased Fréchet Inception Distance (FID) between distributions of high-dimensional representations obtained with: InceptionV3; then a version of InceptionV3 fine-tuned for event classification; and a 3D Convolutional Neural Network that has been specifically designed for event classification. We also normalize the obtained FID values by the FID for two sets of real samples, setting the scores for different representations on the same scale. This novel relative FID metric is used to compare our GAN models to state-of-the-art natural image generative models.
References
The authors intend to publish this work as a journal publication too.
Significance
Fréchet Inception Distance (FID) is the standard metric, in academia and industry alike, used to evaluate natural image generative models. However, to our knowledge, it has only been used once to evaluate models developed by the HEP community. This is because FID was specifically designed for natural image generation applications.
In previous works, HEP-specific neural networks were substituted to InceptionV3 to measure FID. In this work we take the same approach, and measure FID using 2 neural networks trained on HEP data and bring two contributions:
- Rather than the FID for a finite number of samples, we measure the effectively unbiased FID, which was shown to be the real significant metric when it comes to comparing GAN models.
- FID values obtained with different neural networks are not directly comparable. We introduce a normalized FID metric which sets them on the same scale. This allows us to directly compare the performance of our GANs to reference models developed by the industry (e.g., StyleGAN and WGAN-GP)
The evaluation method we present could allow physicists to evaluate generative models more reliably and to directly compare them to the current state-of-the-art natural image generative models, allowing the progress of HEP ML applications to be monitored with respect to that of the industry.
| Experiment context, if any | This work was carried out in the context of the COherent Muon to Electron Transition (COMET) experiment and supervised by Professor Yoshi Uchida. The project was an integral part of the integrated master's in science (MSci) degree at Imperial College London. It builds on the work of the COMET collaboration and the Imperial COMET group but is independent of the collaboration. |
|---|
