Crisis? Surely you must be joking

by Andrea Saltelli (Centre for the Study of the Sciences and the Humanities, University of Bergen, and Open Evidence Research, Open University of Catalonia)

Thursday 7 Jun 2018, 16:30 → 17:30 Europe/Zurich

500/1-001 - Main Auditorium (CERN)

Statistics is experiencing a quality control crisis. There have recently been alarms as to the scientific quality arrangement is several disciplines. The most visible symptom of this possible dysfunction is the so-called reproducibility crisis. In the context of the crisis the discipline of statistics has been going through a phase of critique and self-criticism, due to mounting evidence of poor statistical practice of which misuse and abuse of the P-test is the most visible sign. Most observers have noted that the crisis has technical as well as ethical and behavioural elements which interact with one another – e.g. the ‘publish or perish’ obsession has an impact on selection bias – the tendency to favour positive over negative results. Unlike statistics, mathematical modelling is not a discipline, hence the lack of appropriate internal antibodies to fight a possible infection in the form of quality standards, disciplinary fora and journals and recognized leaders. The main issue in existing practices of mathematical modelling is in the management of uncertainty in model-based inference. Modelling studies can be seen which tend to overestimate certainty, pretending to produce crisp numbers precise to the third decimal digits even in situation of pervasive uncertainty or ignorance. Just as per the case of statistics, no solution is possible without careful appraisal of the social and cultural dimensions of the problem. We suggest that the situation calls an ethics of quantification to be developed, analogous to what is happening in the field of algorithms and big data.

CERNColloquium07.06.pdf