Simpson's paradox: when the parts disagree with the whole

Simpson's paradox describes a trend that shows up in each of several groups of data on its own, then reverses or vanishes once the groups are pooled together. It is not a trick of arithmetic. Both the grouped and the combined numbers are correct. The paradox is that either view, taken alone, can lead to the opposite conclusion.

The best known case is graduate admissions at the University of California, Berkeley, in 1973. Overall, men were admitted at a noticeably higher rate than women, which looked like clear evidence of bias. But statisticians Peter Bickel, Eugene Hammel, and William O'Connell found that within almost every individual department, admission rates for men and women were close, and about as many departments favored women as favored men.

Two traps follow. First, a pooled total can hide the fact that groups differ in size or makeup in a way that drives the overall number, which is what happened at Berkeley, since women disproportionately applied to the most competitive departments. Second, resolving the paradox at the group level does not automatically mean the aggregate figure was wrong to report. Sometimes the aggregate is what matters, and sometimes the breakdown is.

When a striking aggregate statistic is reported, ask whether it holds up once the data are split by an obvious grouping variable, such as department, age group, or region. If the trend reverses within every subgroup, the aggregate number is describing something other than what it appears to describe. Pair any pooled figure with a look at the subgroup that is doing the most work to produce it.