# Introduction to iminuit¶

by Hans Dembinski (maintainer), Max Planck Institute for Nuclear Physics, Heidelberg

We are doing a gentle introduction on how to use iminuit. Feel free to ask any questions!

In [1]:
# basic setup of the notebook

# !pip install iminuit matplotlib numpy
%matplotlib inline
from matplotlib import pyplot as plt
plt.rcParams["font.size"] = 20
import numpy as np


## Simple Fit: line model to scattered $(x, y)$ data¶

• Line model has two parameters $(a, b)$
In [2]:
# let's make a line model
def line(x, a, b):
return a + x * b

a_true = 1.0
b_true = 2.0

# let's make some data
x = np.linspace(0, 1, 10)

# precomputed random numbers from standard normal distribution
z = np.array([-0.49783783, -0.33041722, -1.71800806,  1.60229399,
1.36682387, -1.15424221, -0.91425267, -0.03395604,
-1.27611719, -0.7004073 ])

sigma_y = 0.1 * np.ones_like(x)
y = line(x, a_true, b_true) + sigma_y * z

plt.errorbar(x, y, sigma_y, fmt="o")
plt.xlim(-0.1, 1.1);

• Want to estimate parameters $(a, b)$ of line model from data
• Need score which is minimal when model best agrees with data
• Sum of residuals squared (least-squares method)
• Negated sum of log-likelihood values (maximum-likelihood method)
• MINUIT always minimizes; negate score function to maximize
• Use iminuit to numerically minimize score as function of model parameters
In [3]:
# least-squares score function = sum of data residuals squared
def LSQ(a, b):
return np.sum((y - line(x, a, b)) ** 2 / sigma_y ** 2)

In [4]:
# everything in iminuit is done through the Minuit object, so we import it
from iminuit import Minuit

In [5]:
# create instance of Minuit and pass score function to minimize
m = Minuit(LSQ)

/Users/hdembins/Code/iminuit/py37/lib/python3.7/site-packages/ipykernel_launcher.py:2: InitialParamWarning: Parameter a does not have initial value. Assume 0.

/Users/hdembins/Code/iminuit/py37/lib/python3.7/site-packages/ipykernel_launcher.py:2: InitialParamWarning: Parameter a is floating but does not have initial step size. Assume 1.

/Users/hdembins/Code/iminuit/py37/lib/python3.7/site-packages/ipykernel_launcher.py:2: InitialParamWarning: Parameter b does not have initial value. Assume 0.

/Users/hdembins/Code/iminuit/py37/lib/python3.7/site-packages/ipykernel_launcher.py:2: InitialParamWarning: Parameter b is floating but does not have initial step size. Assume 1.

/Users/hdembins/Code/iminuit/py37/lib/python3.7/site-packages/ipykernel_launcher.py:2: InitialParamWarning: errordef is not given. Default to 1.


• iminuit shows a lot of warnings to make us aware of additional settings
• things may work without, but it is better to set things up properly

### Initial parameter values¶

• MINUIT searches for local minimum by gradient-descent method from starting point
• If function has several minima, minimum found will depend on starting point
• If function has only one minimum, iminuit will converge to it faster if started near minimum
• If no starting value is provided, iminuit uses 0 (which may be bad)
In [6]:
# set start values via keywords for a and b
m = Minuit(LSQ, a=5, b=5)

/Users/hdembins/Code/iminuit/py37/lib/python3.7/site-packages/ipykernel_launcher.py:2: InitialParamWarning: Parameter a is floating but does not have initial step size. Assume 1.

/Users/hdembins/Code/iminuit/py37/lib/python3.7/site-packages/ipykernel_launcher.py:2: InitialParamWarning: Parameter b is floating but does not have initial step size. Assume 1.

/Users/hdembins/Code/iminuit/py37/lib/python3.7/site-packages/ipykernel_launcher.py:2: InitialParamWarning: errordef is not given. Default to 1.



Notice how iminuit figured out that the arguments of LSQ are called "a" and "b". This is a cool gimmick of iminuit.

### Initial step sizes¶

• iminuit computes gradients numerically from finite differences over some step size
• step size should be
• small compared to the curvature of the function $\mathrm{d}^2f/\mathrm{d}a^2$, $\mathrm{d}^2f/\mathrm{d}b^2$
• large compared to numerical resolution (about 1e-14)
• iminuit very tolerant to step sizes and optimizes step size while it is running
• converge rate theoretically faster with optimal step size, but little impact in practice
In [7]:
# set step size with error_<name>=... keyword
m = Minuit(LSQ, a=5, b=5, error_a=0.1, error_b=0.1)

/Users/hdembins/Code/iminuit/py37/lib/python3.7/site-packages/ipykernel_launcher.py:2: InitialParamWarning: errordef is not given. Default to 1.



### "Error definition"¶

• difficult to explain quickly (ask me and I will try), so just remember rule for 1 sigma uncertainties
• errordef=1 for least-squares score function
• errordef=0.5 for maximum-likelihood score function
• only changes uncertainty computation (no effect on minimization)
In [8]:
# set errordef=1 for least-squares score function
m = Minuit(LSQ, a=5, b=5, error_a=0.1, error_b=0.1, errordef=1)

# no more warnings! :-D

# fast alternative: just silence all warnings
m = Minuit(LSQ, pedantic=False)

import matplotlib.image as mpimg
plt.imshow(img);

In [9]:
# check current parameter state (do this at any time)
m.get_param_states()

Out[9]:
 Name Value Hesse Error Minos Error- 0 a 0.0 1.0 1 b 0.0 1.0

### Parameters with limits¶

• Model parameters often have physical or mathematical limits, e.g. $x \ge 0$ in $\sqrt{x}$
• iminuit allows you to set a one-sided and two-sided limit for each parameter
• iminuit will never leave range bounded by limits
• slow convergence for parameters close to limit
In [10]:
# one-sided limit a > 0, two-sided limit 0 < b < 10
m = Minuit(LSQ, a=5, b=5,
error_a=0.1, error_b=0.1,
limit_a=(0, None), limit_b=(0, 10),
errordef=1)

m.get_param_states()

Out[10]:
 Name Value Hesse Error Minos Error- Minos Error+ Limit- 0 a 5.00 0.10 0 1 b 5.00 0.10 0 10

### (Initially) Fixing parameters¶

• for complex model with many parameters, may want to fix some parameters initially
• release fixed parameters when other parameters are close to optimal
• also useful for systematic checks
In [11]:
# fix parameter a
m = Minuit(LSQ, a=2, b=5, fix_a=True,
error_a=0.1, error_b=0.1,
errordef=1)

m.get_param_states()

Out[11]:
 Name Value Hesse Error Minos Error- Minos Error+ 0 a 2.00 0.10 yes 1 b 5.00 0.10

### Covariance and correlation matrix from Hesse¶

In [23]:
# calling hesse explicitly
m.hesse()

Out[23]:
 Name Value Hesse Error Minos Error- 0 a 0.8 1.3 1 b -1.5 2.0 2 c 2.7 0.8 3 d 0.91 0.09
In [24]:
# get full correlation matrix (automatically prints nicely in notebook)
m.matrix(correlation=True)

Out[24]:
 a b c d 1.00 -0.98 0.90 -0.50 -0.98 1.00 -0.96 0.59 0.90 -0.96 1.00 -0.76 -0.50 0.59 -0.76 1.00
In [25]:
# or get covariance matrix
m.matrix()

Out[25]:
 a b c d 1.721 -2.581 0.979 -0.059 -2.581 3.995 -1.593 0.108 0.979 -1.593 0.691 -0.057 -0.059 0.108 -0.057 0.008
In [26]:
# or get matrix as numpy array
m.np_matrix()

Out[26]:
array([[ 1.72054192, -2.58081288,  0.979222  , -0.05947552],
[-2.58081288,  3.99548068, -1.59309437,  0.10762237],
[ 0.979222  , -1.59309437,  0.69138984, -0.05716783],
[-0.05947552,  0.10762237, -0.05716783,  0.00823776]])
In [27]:
# access individual elements of matrices
corr = m.np_matrix(correlation=True)
print(corr[0, 1])
print(corr[0, 3])

-0.9843269379727133
-0.4995753625986588


### Asymmetric uncertainty intervals from Minos¶

In [28]:
m.minos()

Out[28]:
 a Valid Error -1.3 1.3 Valid True True At Limit False False Max FCN False False New Min False False
 b Valid Error -2.0 2.0 Valid True True At Limit False False Max FCN False False New Min False False
 c Valid Error -0.8 0.8 Valid True True At Limit False False Max FCN False False New Min False False
 d Valid Error -0.09 0.09 Valid True True At Limit False False Max FCN False False New Min False False
• Minos can fail, check messages:
• "Valid": everything is chipper
• "At Limit": Minos hit parameter limit before finishing contour
• "Max FCN": Minos reached call limit before finishing contour
• "New Min": Minos found a new minimum while scanning
In [29]:
# Minos errors now appear in parameter table
m.get_param_states()

Out[29]:
 Name Value Hesse Error Minos Error- Minos Error+ Limit- 0 a 0.8 1.3 -1.3 1.3 1 b -1.5 2.0 -2.0 2.0 2 c 2.7 0.8 -0.8 0.8 3 d 0.91 0.09 -0.09 0.09
In [30]:
# plot parameters with errors
v = m.np_values()
ve = m.np_errors()
vm = m.np_merrors()

npar = len(v)
indices = np.arange(npar)

# plot hesse errors
plt.errorbar(indices - 0.05, v, ve, fmt="ob")

# plot minos errors
plt.errorbar(indices + 0.05, v, vm, fmt="or")

# make nice labels
plt.xticks(indices, m.values.keys())
plt.xlim(-0.2, indices[-1] + 0.2)
plt.xlabel("parameter")
plt.ylabel("value");


## Builtin plotting¶

In [31]:
m.draw_mncontour('a','b', nsigma=4);  # nsigma=4 says: draw four contours from sigma=1 to 4

In [32]:
m.draw_profile("a");

In [33]:
m.draw_mnprofile("b");

In [34]:
# get scan data to plot it yourself
px, py = m.profile('a', subtract_min=True)
plt.plot(px, py);