Survivorship bias: the failures you never see

In World War II, the American military studied bombers coming back from raids to decide where to add armor. The returning planes were peppered with bullet holes across the wings and tail, so the obvious move was to reinforce those spots. The mathematician Abraham Wald saw it the other way. The holes showed where a plane could be hit and still limp home. The armor belonged where the returning planes had no holes, around the engines, because the planes hit there were the ones that never came back to be counted.

That is survivorship bias in a single story. We judge by what we can see, and what we can see are the survivors. The failures are gone, invisible, uncounted, and they were carrying half the lesson. Once you know its shape, you spot it everywhere. Successful founders all dropped out of college, says the legend, ignoring the far larger crowd who dropped out and vanished. A supplement worked for everyone in the forum, ignoring everyone it did nothing for, who quietly left the forum.

In health and science it is just as sneaky. A study of people who have taken a drug for years can look wonderful, because the ones who had bad reactions already quit and are not in the sample. A clinic that reports glowing outcomes may simply be one that turns away the hardest cases. The data looks clean. The missing rows are the whole story.

So when a claim is built on winners, stop and ask the Wald question: who is missing? Where are the planes that did not come back, the patients who dropped out, the studies that found nothing and were never published? You usually cannot see them directly. Just remembering that they exist is often enough to stop a shiny result from fooling you.