EWK Covariance discussion 12.7.2018

	•	Participants: 15

	•	Intro
	•	xFitter (Sasha Zenaiev)
	•	2 modes are supported
	•	Covariance matrix
	•	correlations across data sets
	•	nuisance parameters
	•	straightforward for across data sets
	•	data sets are implemented by authors
	•	in simple text format
	•	need to convince experiments to provide correlations
	•	wishes:
	•	both modes in a common format
	•	format is not really important
	•	simple text format preferred
	•	treatment of uncertainties: mulitplcitive or additive
	•	part of common format
	•	provide std chi2 routine ?
	•	Discussion:
	•	xfitter has tool to convert from covariance to nuisance parameters
	•	still need breakdown of systematics
	•	Louie: can one always go back and forth ?
	•	if asymmetric uncertainties ?
	•	exact if only gauss is assumes
	•	nuisance params can always be converted to covariance matrix, but back needs assumptions
	•	HepData (Graeme Watt)
	•	Two representations:
	•	nuisance parameters (preferred)
	•	covariance matrix
	•	Uncertainties (nuisance representation)
	•	at present Root(TGraphAsymmErrors,TH1F), yoda only scatter2d, but can change.
	•	Uncertainties (Covariance representation)
	•	Root(TGraph2DErrors, T2HF) Yoda Scatter3D
	•	Standardize error labels
	•	Covariance matrix: link to Hepdata table
	•	HistFactory used in BSM physics
	•	tools to convert xfitter to HepData files
	•	available from xfitter pyhf 
	•	xfitter: symmetrize with Gaussion
	•	if data tables are stored in yaml or json, then hepdata can deal with it
	•	Nuisance parameters
	•	if it has the same label, then it is assumed to be correlated
	•	for major development personpower is needed.
	•	ask at LPCC and other labs whether support can be provided ?
	•	Rivet & Yoda (Andy Buckley)
	•	progress in HistFactory:  namelist xfitter tables can be converted to Yaml, json  via pyhf
	•	mapping primary data set with covariance matrix
	•	nuisance param: each point contains its own stat and syst uncertainty
	•	difficult to map covariance matrix with data points -> need separate files and links
	•	from individual sources, covariance matrix can be build: C_ij = Sum sigma_i Sigma_j
	•	need standard names to distinguish uncorr from corr uncertainties
	•	Correlations as Yoda metadata
	•	store correlations in header of yoda file for the moment
	•	later: correlate this directly with data points
	•	Discussion
	•	Louie:
	•	yoda output from hepdata 
	•	Recommendation
	•	store bootstrap replica also  as additional tables
	•	ToDo Items:
	•	storing uncertainties: 
	•	uncertainties as nuisance parameter, source and up/down variation for every data point
	•	for stat uncertainties, store also covariance matrix
	•	agreement of names: 
	•	common agreement on correlated and uncorrelated uncertainties
	•	for correlated uncertainties, need only to be consistent within one analysis, since in different analyses uncertainty and source could mean something different.
	•	Tools
	•	pyhf to convert xfitter tables to json files for hepdata
	•	tool needed to convert older correlation tables to json files in hepdata
	•	goodness-of-fit (chi2) for Rivet/yoda histograms and theory comparison
	•	needed for benchmark comparison of EW group