The struggle with p-values

by: in: Researchon April 12, 2016

For most of my professional life I have thought that “statistically significant” results are marked by a p-value of less than 0.05, and that, if the test is used correctly, everything is cool and dandy and can be published. Well…. “if the test is used correctly” is the important consideration here, and it has always been.

The recent warning about the use of p-values by the American Statistical Association (ASA) made things a bit more difficult. The warning was given because of so many misinterpretations published in high impact journals. In my view, the warning resulted in some reasonable reaction and in some over-reaction. Some journals apparently even do not accept papers containing p-values any longer. Digging a little deeper, we find that The Royal Society, as early as 2014, recommended to not use the word “significant”. What now?

Let’s put things into perspective: Working in the field of medical device technology, we know that a medical diagnosis should never be based on a single number. In particular, if a single number crosses a threshold, it is not the end of the world. If a patient’s diastolic blood pressure was 89 mmHg yesterday and it is 90 mmHg today, it does not mean that the patient was healthy yesterday and that she is sick and needs treatment today. The same principle applied to the p-value will result in careful consideration of circumstances, careful description of the methods used, and careful use of the p-value and its interpretation. And that is all the ASA warning tells us: Apply your methods carefully, consider the circumstances, and disclose what you did.

Personally, I am now more careful when it comes to the word “significant”, but I still use it. Furthermore, I am also more careful with p-values, but I keep on using them.

The struggle with p-values

Andreas Spiegelberg

is a Senior Scientist in The Interface Group with long standing experience in medical device development. His main research interest is the biomechanics of the central nervous system, particularly in Normal Pressure Hydrocephalus.

author's page

Leave a Comment

Your email address will not be published. Required fields are marked *


x Thank you for your comment. It will be published after review by the editor.

There are no comments