Why a positive test rarely means what it seems

A base rate is simply how common something is in a population before any new evidence arrives. It sounds like the kind of number a headline would skip, and it usually is. But leaving it out is exactly what turns an accurate test into a frightening, misleading result, because the chance that a positive result means disease depends as much on how rare the disease is as on how good the test is.

Doctors and patients tend to focus on a test's accuracy, often described as sensitivity, how often it catches real cases, and specificity, how often it correctly clears healthy people. A test that is 90 percent sensitive and 91 percent specific sounds close to certain. When researchers asked physicians a version of this problem in 1978, involving a disease with a prevalence of 1 in 1000, most guessed that a positive result meant a 95 percent chance of disease. The correct answer, once the rarity of the disease was factored in, was about 2 percent.

Two traps follow. First, a test's accuracy percentages describe the test, not the person in front of you, and applying them without the base rate confuses the two. Second, the error gets worse as a condition gets rarer, which is exactly when a screening program is most likely to produce far more false alarms than true detections among an otherwise healthy population.

When a study or a screening result cites accuracy figures, ask what the base rate is, meaning how common the condition is in the group being tested, before accepting the headline probability. Restating the same numbers as a natural frequency out of a real group, such as 9 cancers found among 98 positive tests, rather than as a percentage, makes the base rate far harder to ignore. Pair a striking test result with the base rate of the condition it is testing for.