In high energy physics, variable selection and reduction are key to a high quality multivariate analysis. Initial variable selection often leads to a variable set cardinality greater than the underlying degrees of freedom of the model, which motivates the needs for variable reduction and more fundamentally, a consistent decision making framework. Such a framework called PARADIGM, based on a global reduction criterion called the gloss function and relevant to searches for new phenomena in physics, is described. We illustrate the common pitfalls of variable selection and reduction associated with variable interactions and show that PARADIGM gives consistent results in their presence. We discuss PARADIGM’s application to several HEP searches for new phenomena and compare the performance of different measures of relative variable importance, in particular with those based on binary regression. Finally we describe a technique called variable amplification that shows how PARADIGM’s results lead to improved classification performance.