Speaker
Description
Optimization problems in HEP often involve maximizing a measure of the sensitivity of the analysis to an hypothesis with respect to another; the latter is referred to as null hypothesis and in a frequentist framework is tested against the former, which is referred to as alternative hypothesis.
In most cases, it is desirable to fully compute the expected frequentist significance, accounting for all sources of systematic uncertainty and interpreting the result as the real sensitivity of the analysis to the effect sought.
Sometimes, however, either computational or conceptual reasons can favour the use of different or approximate figures of merit.
This work will review the most established frequentist and Bayesian definitions of sensitivity, including approximate significances, and compare them with the fully frequentist significances computed in toy analyses spanning a spectrum of typical HEP use cases. As a part of the comparison, frequentist coverage will be checked for the examined formulas.
Particular care will be taken for the case in which the available data are not enough to satisfy the asymptotic approximations, and the use of the Bartlett corrections to the first-order asymptotic approximation are explored.
