PHYSTAT Seminar: Likelihood-Free Frequentist Inference

by Ann Lee (CMU)

Wednesday 27 Oct 2021, 16:00 → 18:00 Europe/Zurich

The construction of frequentist hypothesis tests and confidence sets with correct coverage has a long history in statistics (Fisher 1925; Neyman 1935), with the equivalence between tests and confidence sets formalized by Neyman. However, in practice, it is often difficult to apply the Neyman construction of confidence sets without relying on large sample asymptotic theory (Wilks 1938). On a related note, scientists have long recognized the importance of checking coverage for constructed confidence sets over all possible parameters, but computationally this has been challenging.In this talk, I will describe our group's recent and ongoing research on developing scalable and modular machine learning based procedures for (i) constructing frequentist confidence sets with finite-sample guarantees of nominal coverage (type I error control) and power, and for (ii) running diagnostics for assessing empirical coverage over the entire parameter space. These methods apply to settings where we have access to a high-fidelity simulator but the likelihood cannot be evaluated and observed data are limited. We refer to the general machinery as "likelihood free frequentist inference". (Part of these efforts are joint with Niccolo Dalmasso, Tommaso Dorigo, Rafael Izbicki, Mikael Kuusela, Luca Masserano, and David Zhao. An earlier version of this work can be found on arXiv:2107.03920)

Ann Lee is a a professor in the Department of Statistics & Data Science at Carnegie Mellon University (CMU), with a joint appointment in the Machine Learning Department. Dr Lee's interests are in developing statistical methodology for complex data and problems in the physical and environmental sciences, especially pertaining to uncertainty quantification and validation in simulator-based inference and prediction based on sequences of satellite image data. She co-directs the Statistical Methods for the Physical Sciences (STAMPS) research group at CMU, and is key personnel in the NSF AI Planning Institute for Data-Driven Discovery in Physics at CMU.

 

 

 

Lee_PHYSTAT_October2021.mp4

Lee_PHYSTAT_October2021.pdf