Pre-registration: locking in the plan before the data arrive

Pre-registration means a study's hypothesis, methods, and planned analysis are written down and filed publicly before the researchers collect or look at their data. The point is to close off a quiet but powerful source of bias: the freedom to try several outcomes, subgroups, or analyses after seeing the results and then report only the ones that turned out significant, while the plan filed in advance draws a clear line between what was tested first and what was found afterward.

A widely cited illustration comes from large, publicly funded heart and lung trials in the United States. Comparing trials completed before and after the year 2000, when prospective registration in a public database became standard practice for this kind of trial, researchers found that the share reporting a significant benefit for the treatment being tested fell sharply, from the majority of earlier trials to a small minority of later ones.

Two traps follow. First, a study that reports a striking result without having pre-registered its hypothesis leaves open the possibility that the reported outcome was chosen, consciously or not, after the researchers had already seen which one looked best. Second, pre-registration does not guarantee a study is well designed or its result is correct. It only guarantees that the plan cannot have been quietly reshaped by the data.

When a study makes a strong claim, look for a pre-registration record, such as a registry entry or a published protocol, filed before the study began, and check whether the outcome being celebrated matches the one that was originally planned. A result that matches its pre-registered plan deserves more confidence than one that emerged from an analysis assembled after the data were already in hand. Pair a striking finding with a look at its registration date relative to when data collection began.