The Field Guide · No. 47
Non-randomized trials: when a trial skips the coin toss
A non-randomized trial is still an experiment, because researchers give the treatment, but people rather than chance decide who goes in which group, or there is no comparison group at all, which makes it easier for a new treatment to look better than it is.
Updated
A randomized controlled trial and a non-randomized trial share the thing that separates both from an observational study: the researchers give the treatment. What differs is how people end up in each group. ClinicalTrials.gov, the US registry of clinical studies, has every trial declare its allocation. Randomized means participants are assigned to groups by chance. Nonrandomized means they are expressly assigned through a non-random method, and the registry's example is physician choice. A third option, not applicable, is for a single-arm trial, where everyone gets the same treatment and the study has no comparison group of its own.
Chance matters because people who choose a new treatment, or are chosen for it, tend to differ from those who are not. The National Cancer Institute defines a nonrandomized trial as one in which participants may pick their group or be assigned by the researchers, and either route lets those differences in. In phase 3 trials, the institute explains, participants are randomly assigned so that any differences are real and not the result of differences in the people in each group.
A single-arm trial still has to compare its patients with someone, and that someone is often a historical control: in the institute's words, an individual treated in the past and used in a comparison group for a study that had no control group. The risk of that comparison was measured in 1982. Henry Sacks, Thomas Chalmers and Harry Smith found six therapies that had been tested both ways. Of 56 trials using historical controls, 44, or 79 percent, found the new therapy better. Of 50 randomized trials of the same therapies, 10, or 20 percent, agreed.
So when a trial makes news, find out whether it was randomized, and if it had a single arm, ask what it was compared with. A non-randomized result can be a fair early signal, and for an effect large enough it can carry real weight. Most of the time it is the start of the case for a treatment, and a randomized trial is what tests it.
The registry's labels
Every interventional study on ClinicalTrials.gov records two design choices that tell you whether, and how, it compared groups: its allocation and its study model. The definitions below are the registry's own.
| Label | Definition |
|---|---|
| Allocation: Randomized | Participants are assigned to intervention groups by chance |
| Allocation: Nonrandomized | Participants are expressly assigned to intervention groups through a non-random method, such as physician choice |
| Allocation: N/A | For a single-arm trial |
| Study model: Single Group | Clinical trials with a single arm |
| Study model: Parallel | Participants are assigned to one of two or more groups in parallel for the duration of the study |
| Study model: Crossover | Participants receive one of two (or more) alternative interventions during the initial phase of the study and receive the other intervention during the second phase of the study |
A single-group trial with allocation marked not applicable has nothing inside the study to compare against. Any claim that the treatment worked rests on a comparison made outside it.
RCT, non-RCT and observational study
The three terms sort on two questions. First, did the researchers give the treatment? If not, it is an observational study. If they did, it is a clinical trial, which ClinicalTrials.gov also calls an interventional study. Second, did chance decide who got what? If so, it is randomized. If not, it is a non-randomized trial.
That makes "non-RCT" a slippery label. In a strict sense it means a trial without randomization. In looser use it lumps those trials together with observational studies. Both lack the protection randomization gives, but they are different designs, and a paper should say which it is.
What the 1982 comparison found, and what it did not
Sacks and his colleagues traced most of the gap to the control groups. Patients in the historical comparisons generally did worse than the control patients in randomized trials of the same therapy, so the new treatment looked better beside them. Adjusting the historical trials for known prognostic factors, where that was possible, did not appreciably change the results. The authors concluded that biases in patient selection may irretrievably weight such trials in favor of new therapies.
They did not let randomized trials off. The same paper warned that randomized trials may miss clinically important benefits when too little attention goes to sample size, the problem statistical power describes. Evidence rankings leave room for exceptions too: the Oxford Centre for Evidence-Based Medicine places historically controlled studies at level 4 for treatment benefits, below randomized trials, but puts an observational study with a dramatic effect at level 2.
What to remember
- A non-randomized trial gives a treatment like an RCT does, but people, not chance, decide the groups, or there is only one group.
- Trials compared with historical controls found the new therapy better far more often than randomized trials of the same therapies: 79 percent against 20 percent in one 1982 analysis.
- A non-randomized trial is not an observational study; check which one a "non-RCT" label means, and what a single-arm trial was compared with.
From the record
Forty-four of 56 HCTs (79 percent) found the therapy better than the control regimen, but only 10 of 50 RCTs (20 percent) agreed.
Asked often
What is the difference between an RCT and a non-RCT?
In a randomized controlled trial, chance decides who gets which treatment. In a non-randomized trial the researchers still give the treatment, but people are placed in groups another way, such as by physician choice, or everyone gets the same treatment. Without randomization the groups can differ in ways that get credited to the treatment.
What is a single-arm trial?
A trial in which every participant receives the same intervention, so there is no comparison group inside the study. ClinicalTrials.gov records it as a single-group study model with allocation not applicable. Its results have to be judged against an outside comparison, often patients treated in the past.
Is a non-randomized trial the same as an observational study?
No. In a non-randomized trial the researchers assign and give the treatment; in an observational study they only record what people were already doing. The two share the lack of randomization, which is why both are weaker than an RCT for showing cause.
What is a historical control?
The National Cancer Institute defines a historical control subject as an individual treated in the past and used in a comparison group when researchers analyze a study that had no control group. A 1982 analysis found trials using historical controls reported benefit far more often than randomized trials of the same therapies.
Read the news better, every morning.
We send you the day's real progress in science, medicine and beyond, told plainly and sourced. Free, forever.
Free for good. One click to leave.