Spacing statistics: what they are and how to use them

by Lolian Shtembari (MPIM)

Wednesday 9 Nov 2022, 16:00 → 17:30 Europe/Zurich

The use of spacings between ordered real-valued numbers is very useful in many areas of science. In particular, either unnaturally small or large spacings can be a signal of an interesting effect. As particle physicists, we are interested in the appearance of the unexpected clustering of  values, indicating the presence of a new process, or large gaps between the ordered values, allowing us to set upper limits on the normalization of a distribution. Order statistics have been studied in great depth in the statistics community, but the work is poorly known in the physics community.  We introduce new statistical tests based on the observed spacings of ordered data. These statistics are sensitive to detect non-uniformity in  random samples, or short-lived features in event time series. Under some conditions, these new test can outperform existing ones, such as the well known Kolmogorov-Smirnov or Anderson-Darling tests, in particular when the number of samples is small and differences occur over a small quantile of the null hypothesis distribution. Additionally, we discuss tests aimed at setting limits on the signal event rate in experiments contaminated with a background too poorly understood to subtract. Finally, we provide a detailed description of all the tests presented, discussing their derivation and parametrisation and presenting examples and proposed applications for the analysis of neutrino experiments.

 

Relevant literature:

https://arxiv.org/abs/2008.02048
https://arxiv.org/abs/2111.02252
https://iopscience.iop.org/article/10.1088/1475-7516/2022/10/024
 

 

GMT20221109-150218_Recording_3440x1440.mp4

PHYSTAT_Spacings_statistics_Lolian_Shtembari.pdf