Dr Maria Grazia Pia (INFN Genova)
The Statistical Toolkit provides an extensive collection of algorithms for the comparison of two data samples: in addition to the chisquared test, it includes all the tests based on the empirical distribution function documented in literature for binned and unbinned distributions. Some of these tests, like the Kolmogorov-Smirnov one, are widely used; others, like the Anderson-Darling or the Cramer-von Mises tests, have been previously used in more sophisticated physics data analyses; nevertheless, several of the tests in the Statistical Toolkit are largely unknown in high energy physics applications, and a few of them, like the weighted formulations of the Kolmogorov-Smirnov test, are available for the first time in an open-source software tool for statistical analysis. No systematic evaluation of the power of the goodness-of-fit tests has been documented so far in literature. The present work presents a comprehensive study of the power of all the goodness-of-fit tests in rich collection of the Statistical Toolkit in a variety of use cases specific to high energy physics experiments; it highlights the relative merits and deficiencies of the algorithms to deal with peculiar characteristics of the configurations under study. The results of this study provide guidance for the selection of the most appropriate algorithm in experimental analyses. The Statistical Toolkit user layer is interfaced to AIDA-compliant analysis tools and to ROOT.