One of the most common applications of machine learning in high energy physics is in event selection (and categorization). The physics goals of event selection and categorization are to improve the significance of a potential excess (for signal discovery/upper limit setting analyses), and to reduce the uncertainty of a parameter measurement (parameter measurement analyses).
Event selection using machine learning is based on the "signal is better than background" heuristic. While it is clear how the heuristic would help with the physics goals, it turns out that they are not completely aligned. In fact, certain signal events could be worse for the sensitivity of an analyses than certain background events.
In this talk we will provide optimal event selector and categorizer training prescriptions designed to maximize the expected statistical significance of an excess (by changing how ML outputs are used), and minimize the statistical uncertainty of a measurement (by changing the supervisory signal used in training the ML algorithms). Along the way, we will point out exactly how our methods realign the goals of event selection and categorization with the physics goals. Finally, we will indicate how our method can be extended to minimize the systematic uncertainties in parameter measurements as well.