PHYSTAT Dark Matter 2019

31 Jul 2019, 00:30 → 2 Aug 2019, 22:45 Europe/Stockholm

Stockholm university

You can find announcements and last minute information on our twitter channel: https://twitter.com/phystat

Videos of some of the talks can be found on this website: http://video.fysik.su.se/DarkMatter2019.html

Rationale:

The direct detection of dark matter poses specific challenges to statistical inference mainly related to low-number statistics and difficult to model signal and backgrounds. These challenges have led to the development of methods including Bayesian inference for background modelling, varieties of frequentist treatments of nuisance parameters for confidence intervals and hypothesis testing and different blinding protocols.
In addition, direct detection of dark matter is a field with a large number of experiments. Inter-experiment comparison of results motivates an attempt to agree on a few fiducial choices for statistical conventions. Consequently the proposed workshop will have two aims:

1. present and discuss the various statistical methods used in direct detection experiments and
2. work towards a white paper recommending a technique (or set of techniques) that can produce statistically sound exclusion limits or detection claims, and will allow straight forward inter-experiment comparison. 

The programme of the workshop is currently under development. It will feature invited and contributed talks on a variety of subjects.

 

Social activities:

The dinner will consist of a 4 hour cruise and three course meal in the beautiful Stockholm Archipelago. 

The workshop is supported by the following institutions:

  

Dinner_cruise_map_v01.pdf

Group_picture_PHYSTAT2019.JPG

map_AlbaNova.pdf

map_lunch.pdf

Phystat_poster.pdf

white_paper.pdf

Wednesday, 31 July

General: Preamble and Intro to Statistics

Generic session block

Convener: Jan Conrad

1 Introductory Statistics Talks

These will review some simple statistical concepts that are relevant to this Workshop. Among other topics, it will include upper limits, p-values and likelihood ratios. It is intended for those who would like to be reminded of their Statistics, before the Workshop begins.

Speakers: Glen Cowan, Louis Lyons (Imperial College (GB))

cowan_phystatdm_intro_31jul19.pdf

Introductory_Statistics_for_PHYSTAT-DM2019_v2.pdf

Introductory_Statistics_for_PHYSTAT-DM2019_v2.pptx

11:00 Break

General: Intro to Workshop

Generic session block

Convener: Hugh Lippincott (Fermilab)

2 Intro to workshop

Intro to workshop

Speaker: Jan Conrad

Phystat-DM_intro.v2.pdf

3 Status of direct detection experiments and their approaches to statistical inference

In this talk I will give an overview of the current status of recent and future direct detection experiments and outline the practices that have been adopted for statistical inference from data and for making sensitivity projections. I will then discuss why a common approach to the choice of test statistic, the procedure for switching from a discovery mode to signal characterization and the p-value required for discovery would be beneficial to the direct detection community.

Speaker: Dr Jim Dobson

DD-Status-Dobson-PHYSTAT-DM-2019.pdf

4 Including direct detection likelihoods in global fits

Direct detection experiments place some of the strongest constraints on models of dark matter and are therefore essential to include in global analyses of such models. In this talk I will discuss the specific requirements for and possible applications of likelihood functions from direct detection experiments. I will give a brief introduction into the public code DDCalc, which provides approximate direct detection likelihoods for arbitrary interactions between dark matter and nuclei, and present some of the recent results.

Speaker: Felix Kahlhoefer (RWTH Aachen)

Phystat.pdf

12:40 Lunch

General: Blind Analysis, Look Elsewhere

Generic session block

Convener: Tina Pollmann (Technische Universität München)

5 Blinding and salting in dark matter searches.

Direct detection experiments rely on a variety of bias mitigation strategies, most notably blinding and salting. I will review the main challenges and methods for blinding and salting, in preparation for the next generation of dark matter searches.

Speaker: Maria Elena Monzani (Stanford University)

PHYSTAT-2019-Monzani.pdf

6 Blinding Strategies for Dark Matter Searches

A variety of data blinding strategies have been adopted by different experiments searching for dark matter. I will present a brief overview of the relative strengths and weaknesses of some of the most common strategies, and describe the reasoning for and implementation of these methods by some leading dark matter experiments.

Speaker: Dr Ben Loer (Pacific Northwest National Laboratory)

Loer_DMBlinding_split43.pdf

7 Correcting for the look-elsewhere effect: why, when and how

The look-elsewhere effect is a phenomenon which often arises when looking for signals whose location is not known in advance. In this setting, signal searches can be conducted by performing several tests of hypothesis at different positions over the search area considered. However, if the result of each individual test is not adequately adjusted for the fact that many tests are conducted simultaneously, the overall probability of false discoveries rapidly increases with the number of tests. Alternatively, one can consider the unknown position of the signal as a nuisance parameter and construct confidence intervals and statistical tests of hypothesis by means of Monte Carlo simulations or methods relying on random fields and extreme value theory. The goal of this talk is to provide an overview of the most common methods used to correct for the look-elsewhere effect and highlight their advantages and limitations with respect to the goal of the experiment and the conditions under which the statistical analysis is performed.

Speaker: Dr Sara Algeri (University of Minnesota)

Presentation slides

16:00 Break

General: Experience Outside Dark Matter, and Limits

Generic session block

Convener: Jim Dobson

8 The LHC Experience

The ATLAS and CMS collaborations have produced numerous results during the
first two data-taking runs of the LHC, ranging from precision measurements of SM processes to searches for exotic phenomena and the dicovery of the Higgs boson. All of these results make use of sophisticated statistical techniques, not only to provide statistical inferences from the data, but also during the development and review of the data analysis for optimisation and validation of the results. In this talk, I will cover some of the common practices and tools used at both ATLAS and CMS, and practical aspects of producing statistical results, including combinations across different channels or experiments.

Speaker: Nicholas Wardle (Imperial College (GB))

practical_aspects_nwardle_v2.pdf

9 Statistical Inference in Double-Beta Decay Searches

The neutrinoless double-beta decay is a hypothetical nuclear transition predicted by most of the theories that explain the origin of neutrino masses or the dominance of matter over antimatter in our Universe. The primary experimental signature for this transition is an excess of monoenergetic events. While the statistical problem can be traced back to the simple search for a peak over some background, new challenges arise because of the extremely low counting rates expected in the next-generation experiments, in which a single signal count could lead to a discovery. The methods used in the field to evaluate the sensitivity of an experiment, handle the systematic uncertainties and derive constraints on the signal strength will be discussed.

Speaker: Matteo Agostini

agostini-0vbb-phystat-dm.pdf

10 Limit setting: evolution vs. inteligent design

In this talk, I will review the various methods, tools, tricks, and shortcuts used by various dark matter experiments to set upper limits on dark matter interaction cross-section. Focusing on the progression of the methods as the number of parameter increases and the background decreases I will discuss the big question of evolution vs. intelligent design (in the statistical inference realm J).

Speaker: Dr Hagar Landsmann (Weizmann Inst. of Science)

stat_evolution_hagarLandsman.pdf

Poster session: Poster Session and Reception

11 Bayesian and frequentist approaches to discoveries

I discuss findings from my recent comparison of Bayesian and frequentist approaches to discoveries (1902.03243). I introduce a counting experiment in which we are searching for a signal in the presence of a background, from which I generate pseudo-data. With that pseudo-data, I contrast the evolution of the $p$-value and posterior as we accumulate data and directly compare global $p$-values and the posterior of the background model. I find that in this toy problem $p$-values are typically smaller than the posterior by one or two orders of magnitude. I discuss the relevance of my findings to direct detection experiments and suggest similar studies to check the behavior of our statistical approaches in that context.

Speaker: Dr Andrew Fowlie (Nanjing Normal University)

12 DAMA/LIBRA Phase I and II combined analysis in the standard WIMP scenario

In 2018 DAMA/LIBRA has reported results about 6 more annual cycles, the so-called Phase-II, which follows an upgrade of the detector with respect to Phase-I. A combined frequentist analysis of the results of DAMA Phase I and II is presented. The combined analysis is compared with each individual Phase result with the same assumptions. A discussion of nuisance parameters is reported.

Speaker: Dr Ianni Aldo (INFN - LNGS)

13 Improving the LUX PLR Code

LUX uses the hypothesis testing based on a profile likelihood ratio to determine the consistency of its data to a background only hypothesis, as well as to set confidence limits on the interaction strength of potential dark matter candidates.
In this talk I will present on improvements made to the code base used for conduction these tests. The model generation is now streamlined based on configuration files generated by a newly developed python tool. This allows analysis specific changes to be easily implemented without a substantial re-write of the model generation code. Things that can now be easily altered include: combining multiple runs of the experiment, altering which observables are used (eliminating the spatial variables, or including a machine learning WIMP-score, for instance), the energy range of the analysis, or which backgrounds are included. Additionally, we have introduced additional hypothesis test inversion methods custom built for simplicity and efficiency.

Speaker: SHAUN ALSUM (University of Wisconsin-Madison)

14 Statistical Analysis for COSINE-100

COSINE-100 is an experiment to detect dark matter induced recoil interactions in NaI(Tl) crystals in order to test the DAMA/LIBRA collaboration’s claim. The COSINE-100 detector has been operating since September of 2016 in the 700-m-deep Yangyang underground laboratory. First shape  and annual modulation analyses have been completed, and the related statistical approaches will be presented. In addition, we introduce our own analysis tool based on bayesian statistics for several other analyses in progress.

Speaker: Dr Young Ju Ko (Institute for Basic Science (IBS))

15 Uncertainties in the dark matter velocity distribution

I introduce a new formalism for incorporating uncertainties in the velocity distribution of dark matter in direct detection experiments (1809.02323). The method constructs a prior over upon possible velocity distributions. The prior penalizes departures from our expectation (e.g., a Maxwellian) according to the relative entropy. The uncertainty is subsequently marginalized using an exact result. We apply this formalism to results from XENON1T. Lastly, we discuss problems with this approach and possible ways forward.

Speaker: Dr Andrew Fowlie (Nanjing Normal University)

16 Using the profile-likelihood method to search for dark matter in DEAP-3600

The DEAP-3600 detector based 2km underground at SNOLAB (Sudbury, Canada) is a dark matter direct detection experiment. The detector consists of a single-phase liquid argon (LAr) target, of 3279 kg mass. Currently, there have been two WIMP dark matter searches performed by the DEAP-3600 collaboration; for both results, a cut-and-count approach was employed. In this talk, the development of a profile-likelihood ratio statistical test and its application to DEAP-3600 will be presented. This test allows the WIMP search to account for the expected distribution of WIMPs and backgrounds in a multi-dimensional parameter space, and thereby perform a more sensitive search. Furthermore, we will also show how the profile-likelihood approach can be used to search for hidden photons and axion-like particles in the DEAP-3600 detector. We will discuss the expected signature from such particles, and how this approach can be used to search for them over the naturally present backgrounds.

Speaker: Ms Ashlea Kemp (Royal Holloway, University of London)

17 WIMP Dark Matter Modulation Analysis on Data from Low Threshold Germanium Detectors

we present results on light weakly interacting massive particles (WIMPs) searches with annual modulation (AM) analysis on data from a 1-kg mass p-type point-contact germanium detector of the CDEX-1B experiment at the China Jinping Underground Laboratory. Data set with a total live-time of 3.2 years within a 4.2-year span are analyzed with physics threshold of 250 eVee. Annual modulation limits on WIMP-nucleus (chi-N) elastic scatterings cross-section through spin-independent interactions, as well as limits on Migdal effects, bremsstrahlung emissions of WIMP-nucleus will be explained. Diurnal modulation limits on WIMP-nucleus and annual modulation limits on WIMP-electrons (chi-e) will also be reported. In these analyses, we adopted a binned chi-squared approach, that allowed us to avoid background interpretation. Unknown background contributions, as well as those from time static dark matter contributions, are treated as free parameters to be fitted.

Speaker: Hau-Bin Li (Academia Sinica, Taipei, Taiwan)

Thursday, 1 August

General: Limit Setting Methods

Generic session block

Conveners: Ben Loer, Ben Loer (Pacific Northwest National Laboratory)

18 Likelihood asymptotics and beyond

I will review some classical methods of asymptotic inference and their higher order extensions. The focus will be on modern likelihood based solutions, though Bayesian counterparts will be mentioned in by-passing. The discussion will touch upon topics such as small sample sizes, large number of nuisance parameters, nonregular settings and complex models.

Speaker: Alessandra Brazzale

PhyStatDM_Brazzale_talk.pdf

19 Improved Inference for the Signal Significance

We present a method for deriving signal significance p-values in the 5 sigma region for finite samples, to order O(n^{-3/2}), for a number of signal detection statistics (Wald, score, and likelihood ratio). Connection with the look elsewhere effect is discussed.

The talk is based in part on the article by I. Volobouev and A. Trindade 2018 JINST 13 P12011

Speaker: Igor Volobouev (Texas Tech University (US))

asymptotics.pdf

20 Imposing overcoverage on small signals.

An experiment reporting only upper limits must allow the exclusion of arbitrarily low signals, and even the no-signal case if the confidence intervals are to cover.
Many collaborations elect to avoid this, for example by using the CLs method to penalise downwards fluctuations, or by imposing a threshold below which signals are not excluded. In my presentation, I will review the two approaches, and discuss them from the perspective of a direct-detection experiment. Lastly, I will also mention the related issue of using different significance thresholds for confidence intervals and excess claims.

Speaker: Dr Knut Dundas Morå

mora_overcoverage_presentation_1.pdf

10:30 Break

General: Signal and Backgrounds

Generic session block

Convener: Olaf Behnke (DESY)

21 Detecting new signals under background mismodelling

When searching for new astrophysical phenomena, uncertainty arising from background mismodelling can dramatically compromise the sensitivity of the experiment under study. Specifically, overestimating the background distribution in the signal region increases the chances of missing new physics. Conversely, underestimating the background outside the signal region leads to an artificially enhanced sensitivity and a higher likelihood of claiming false discoveries. The aim of this work is to provide a unified statistical algorithm to perform modelling, estimation, inference and signal characterization under background mismodelling. The method proposed allows to incorporate the (partial) scientific knowledge available on the background distribution, and provides a data-updated version of it in a purely nonparametric fashion, without requiring the specification of prior distributions. If a calibration sample or control regions are available, the solution discussed does not require the specification of a model for the signal; however, if the signal distribution is known, it allows to further improve the accuracy of the analysis and to detect additional signals of unexpected new sources.

Speaker: Dr Sara Algeri (University of Minnesota)

Presentation slides

22 Partially specified models and nuisance parameters

I will speak about two aspects of partially specified models. The first of these arise from modelling explicitly only aspects of direct concern, retaining a degree of agnosticism over other aspects of the data generating process. I will illustrate with examples how this leads to a large number of nuisance parameters, and how these can be evaded in some situations. The second type of partially specified model comes from an explicit assumption of sparsity. For instance in regression problems with a large number of potential explanatory variables, it is sometimes natural to assume that relatively few of these have a real effect. It is not known which ones, which makes the model itself as well as the parameters of that model the object of inference. If scientific understanding is the goal rather than prediction, then one should aim for a confidence set of models: all effective simple scientific explanations that are compatible with the data at a given level.

Speaker: Dr Heather Battey (Imperial College London)

DarkMatter20190730.pdf

Discussion

Convener: Olaf Behnke (DESY)

Discussion Astro.pdf

white_paper.pdf

12:30 Lunch

General: Signal and Backgrounds

Generic session block

Conveners: Maria Elena Monzani (SLAC National Accelerator Laboratory), maria monzani (Stanford University)

23 Crystal-based Detector Background and Response Modeling

In order for an experiment to be able to claim discovery of a signal, it must first master its backgrounds and understand how a signal will manifest in the detector. The range of energies involved in the interactions tend to be low due to kinematics so much of an experiment's sensitivity is driven by its threshold. In this region, modeling the detector response can be difficult but I will describe past approaches and on-going work in this area. For experiments, control of backgrounds can be achieved in detector construction by using ultra-pure materials however this is becoming difficult to achieve as experiments improve in sensitivity. As experiments exchange event-by-event discrimination power for lower thresholds, the remaining backgrounds can become significant and searches are no longer background free. The challenge becomes to model these backgrounds so that cuts can be optimized and limits can be set using likelihood-based techniques. I will discuss approaches taken by experiments to understand these backgrounds and model the detector response. Lastly, I will highlight challenges I see in the foresee in the future of direct detection experiments in regards to these two areas.

Speaker: Robert Calkins (Southern Methodist University)

calkins_PhyStat2019.pdf

24 Background and detector response modeling in single electron-hole pair sensitive crystal detectors

We have entered the era of single electron-hole pair sensitive crystal detectors with a threshold as low as the indirect band gap. These detectors are excellent devices to search for light dark sector particles with masses well below the threshold of to date typical direct Dark Matter search detectors. But with new opportunities come new challenges. New sources of background govern the accessible energy range and the quantized nature of the detector response becomes a vital part of the response model. Without a profound understanding of both background and detector response searches with these detectors will remain without discovery potential. Their modeling is thus a very active field of research. This talk will discuss the current status and the challenges of background and response modeling for ongoing single electron-hole pair sensitive crystal experiments using silicon detectors.

Speaker: Belina von Krosigk (University of Hamburg)

2019_PhystatDM_BvKrosigk.pdf

25 A statistical model for the leakage of backgrounds mitigated by pulseshape discrimination methods

In many types of scintillator-based dark matter experiments, pulse shape discrimination (PSD) is used to mitigate backgrounds. The leakage probability of events mitigated through PSD into the region of interest (ROI) is an important parameter used to define the ROI and to inform the ROI background model. Determining the leakage probability requires an understanding of the distribution of backgrounds in the PSD parameter. This is especially important in liquid argon based experiments, where electromagnetic background in recent and planned experiments requires PSD to remove 10ˆ7 to 10ˆ9 events from the energy ROI. As experiments push down the energy threshold, the distribution of signal events in the PSD parameter can no longer assumed to be Gaussian. Thus, properly modelling the signal event distribution in the PSD parameter also becomes important not just to understand the background but also to locate the ROI.
We present a physics-based statistical model to describe the distribution of events in the common prompt-fraction-based PSD parameters. The model can be used for different types of backgrounds as well as to describe the distribution of signal events.

Speaker: Tina Pollmann (Technische Universität München)

2019PhystatPollmann.pdf

15:30 Break

General: Backgrounds/Response Modeling

Generic session block

Conveners: Jelle Aalbers (Nikhef), Dr Jelle Aalbers (Stockholm University)

26 A novel approach to assess the impact of the Fano factor on the sensitivity of low-mass dark matter experiments

The statistical fluctuation of the number of e-/ion pairs produced in an ionizing interaction is known to be sub-Poissonian, the dispersion being reduced by the so-called “Fano Factor”. Due to a lack of appropriate modelling tools, this phenomenon is often folded into the overall energy response of ionization detectors. While this may be adequate down to relatively low-energies, this treatment is not sufficient for a new generation of dark matter experiments with single-quanta sensitivity, such as the NEWS-G light dark matter experiment. To address this problem, we propose the use of the Conway-Maxwell-Poisson distribution for the purpose of modelling primary ionization (D. Durnford et al., Phys. Rev. D 98, 103013 [2018]). This choice of distribution is supported by calibration with sub-keV Ar-37 x-rays and a UV laser in a NEWS-G Spherical Proportional Counter (Q. Arnaud et al., Phys. Rev. D. 99, 102003 [2019]). The potential impact of the Fano Factor on sensitivity to low mass dark matter is discussed as well.

Speaker: Mr Daniel Durnford (University of Alberta)

DDurnford_PHYSTAT.pdf

Discussion: Discussion 2

Conveners: Jelle Aalbers (Stockholm University), Jelle Aalbers (Nikhef)

Discussion Astro.pdf

white_paper.pdf

18:00 Workshop dinner

Friday, 2 August

General: Nuisance Parameters (Form Factors/Halo)

Generic session block

Convener: Luca Grandi (The University of Chicago)

27 Nuclear physics of dark matter direct detection

I will give an overview of the nuclear physics input required for the interpretation of direct detection searches for dark matter, concentrating on approaches based on chiral effective field theory.
In particular, I will discuss the sources of uncertainty in the calculation of the nuclear responses, to trigger the discussion if and how to include these uncertainties in statistical analyses.

Speaker: MARTIN HOFERICHTER (University of Washington)

Talk_Hoferichter.pdf

28 Dark matter in the Milky Way

The Gaia satellite is transforming our understanding of the distribution of dark matter in the Milky Way. I’ll discuss two structures that have recently been detected in the Milky Way, the S1 stellar stream and the Gaia Sausage, and their impact on experiments searching for the direct detection of dark matter.

Speaker: Dr Christopher McCabe (King's College London)

McCabe_Phystat.pdf

Discussion

Convener: Luca Grandi (The University of Chicago)

Discussion Astro.pdf

white_paper.pdf

10:30 Break

General: Machine Learning

Generic session block

Convener: Hagar Landsmann

29 Machine learning in HEP

Machine Learning techniques have been employed in high energy physics already for decades, and have grown to be an important ingredient to event processing. This talk will focus on examples of recent applications and dedicated developments of advanced deep neural networks for particle reconstruction, identification and regression, as well as global event classification. Moreover, the talk will shortly discuss possibilities to mitigate data-to-simulation differences with machine learning techniques.

Speaker: Jan Kieseler (CERN)

jk_ML_in_HEP_20190802.pdf

30 Dark Machines - using machine learning to solve dark matter related problems

Dark Machines is a research collective of about 200 physicists and data scientists, who use state-of-the-art machine learning techniques to solve dark matter related problems. These problems are typically organised as challenges: physicists provide datasets and the machine learning experts try to solve the problem in the best possible way. A few of the current focuses are particle track reconstruction, analysing full sky gamma-ray for detecting, localising and classying point sources, parameter estimation for strong gravitational lensing and collider searches for new physics using machine learning. Additionally, we are building data visualisation tools and storage tools that help the community sharing their work. In this talk I will give an overview of the challenges and the machine learning methods that are used to solve them,

Speakers: Luc Hendriks (Nikhef), Luc Hendriks

Darkmachines Stockholm.pdf

12:00 Lunch

31 Reflections on 20+ years of Feldman-Cousins: Hypothesis testing of a point null vs a continuous alternative

I will discuss aspects of the frequentist and Bayesian approaches to testing a point null hypothesis (say mu=0) versus a continuous alternative hypothesis (say mu>0). This test arises frequently in particle physics (including dark matter searches), where mu is the signal strength. The frequentist testing approach maps identically onto the frequentist theory of confidence intervals. Thus, as Feldman and Cousins eventually realized, the method advocated in their 1998 paper on confidence intervals maps identically onto the "classical" theory of hypothesis testing in Kendall and Stuart (which in addition includes nuisance parameters). Meanwhile, the traditional Bayesian approach to hypothesis testing (due to Jeffreys) is completely separate from the Bayesian approach to credible intervals, with no corresponding mapping. Direct sensitivity to the prior pdf for mu, even in the asymptotic limit of large sample size, is a consequence, as is the Jeffreys-Lindley paradox (arXiv:1310.3791). My talk will draw on parts of my “Lectures on Statistics in Theory: Prelude to Statistics in Practice” arXiv:1807.05996.

Speakers: Robert Cousins Jr (University of California Los Angeles (US)), Robert Cousins Jr (University of California Los Angeles (US)), Robert Dacey Cousins Jr (Univ. of California Los Angeles (UCLA))

cousins_phystat-DM_stockholm_2019.pdf

Discussion: White Paper/Discussion 3

Conveners: Hugh Lippincott (Fermilab), Jan Conrad

Discussion Astro.pdf

white_paper.pdf

15:10 Break

General: Summaries

Generic session block

Convener: Jan Conrad

32 Statisticians summary

Statisticians summary

Speaker: Alessandra Brazzale

PhyStatDM_summary_Brazzale.pdf

33 Physicists Summary

Physicists Summary

Speaker: Hugh Lippincott (Fermilab)

PhystatDM_2019_Lippincott.pdf

Discussion: Optional: White Paper 2

Discussion Astro.pdf

white_paper.pdf