What a p-value really tells you (and what it doesn't)

The p-value is one of the most cited and most misunderstood numbers in science. In plain terms, it measures how surprising your data would be if there were actually no real effect at all. A small p-value means the data would be unlikely to look this way by pure chance, so researchers take that as a hint that something real might be going on.

The common cutoff is 0.05, and results below it get called "statistically significant." But that threshold is a convention, not a law of nature, and this is where people go wrong. A p-value is not the probability that the finding is true, and it is not the probability that the result is due to chance. It only speaks to how compatible the data are with the idea of no effect.

Two traps follow from this. First, a significant p-value says nothing about how big or important an effect is. A trivial difference can be significant in a large study. Second, a p-value just above 0.05 does not mean nothing is there, and one just below it does not mean something definitely is. Treating 0.05 as a magic on-off switch is exactly the mistake the statisticians who invented the tool warn against.

So when a headline leans on the word significant, remember what it does and does not mean. It suggests the pattern is probably not pure luck. It does not tell you the finding is certain, large, or important. Pair the p-value with the effect size and whether the result has been replicated, and you will read studies far more accurately than the headlines do.