How Do You Synthesize Evidence When Studies Use Different Comparison Groups?

What if studies evaluate the same intervention against different alternatives?

You find ten studies evaluating the same intervention and measuring the same outcome. At first glance, they appear ideal for synthesis. Then you inspect the comparison groups.

Some compare the intervention with no treatment. Others use placebo, usual care, a minimal intervention, or an active alternative. Even “usual care” differs among studies.

Should all ten effect estimates be combined?

Not automatically. An intervention effect is inherently comparative. The effect of A versus no treatment is not necessarily the same quantity as the effect of A versus another effective treatment. Changing the comparator can change both the magnitude and the practical meaning of the estimated effect.

Ask what each intervention is being compared against before pooling

Every effect estimate is defined partly by its comparator

There is no intervention effect without a comparison

When a study reports that an intervention reduced an outcome by a certain amount, that effect is shorthand for a contrast. Reduced compared with what?

Consider an intervention A. A trial comparing A with no intervention estimates A versus no intervention. A trial comparing A with treatment B estimates A versus B. Even with the same population and outcome, these are different contrasts.

This sounds obvious when written explicitly, yet comparator differences are easily hidden once studies are reduced to rows of effect sizes in meta-analysis software.

Different comparator types can imply different research questions

Common comparison conditions include no intervention, waiting list, placebo or sham intervention, usual care, attention control, minimal intervention, and another active intervention. These controls do not necessarily represent equivalent counterfactual conditions.

The table provides useful categories, but the label should never replace inspection of the actual intervention content.

“Usual care” is not necessarily one intervention

Usual care deserves particular scrutiny. In one study, it may mean occasional monitoring. In another, participants may receive counseling, medication, educational materials, or access to specialist services. Usual care can also change over time as professional practice changes.

Pooling all “usual care” comparisons assumes that those conditions are similar enough that their differences do not materially alter the intervention effect being estimated. Sometimes that assumption is reasonable. Sometimes it is decidedly not.

The same issue applies to labels such as “standard teaching,” “routine support,” “business as usual,” and “control.” Extract what participants actually received rather than relying solely on the authors' shorthand.

Comparator choice can modify the observed effect

Suppose intervention A produces an average improvement of 10 units compared with no intervention. If treatment B already produces an 8-unit improvement, the contrast between A and B might be much smaller.

That does not make either result contradictory. The studies answer different comparative questions.

Decide whether comparators can reasonably be grouped before seeing the results

Ideally, your review protocol should define which comparator categories are eligible and which may be combined. The decision should be based on substantive reasoning about the interventions, not on whether grouping them produces a convenient meta-analysis or a more attractive result.

Cochrane guidance emphasizes defining how eligible studies will be grouped for synthesis. In some contexts, placebo and control groups may be sufficiently similar to combine, particularly when differences in care are unlikely to matter. In other contexts, those differences can be important.

Ask whether comparator differences could plausibly modify the relative effect. Consider treatment intensity, access to other services, participant expectations, contact time, implementation, setting, and other characteristics relevant to the intervention.

Different comparators can create apparent heterogeneity

If studies using weak controls show large effects while studies using active comparators show smaller effects, statistical heterogeneity may partly reflect the comparison being made rather than random variation among studies.

Simply fitting a random-effects model does not resolve this conceptual difference. A random-effects model allows underlying effects to vary, but you still need to ask why they vary and whether an average across those contrasts answers a useful question.

Comparator type may therefore warrant separate meta-analyses, subgroup analysis, meta-regression where appropriate, or a structured narrative synthesis. The analytical choice depends on the evidence base and should not be determined solely by a heterogeneity statistic.

Never compare isolated treatment arms across separate randomized trials as though they were randomized against each other

Suppose some trials compare A with C and other trials compare B with C. You want to know whether A is better than B.

One tempting approach is to take outcomes from the A arms of the first trials and compare them directly with outcomes from the B arms of the second trials. That discards the protection provided by within-trial randomization. Differences between the A and B groups may reflect differences between the studies rather than differences between the interventions.

Cochrane explicitly warns against this form of naive indirect comparison.

Indirect comparisons require a common comparator and additional assumptions

When A and B have never been compared head-to-head but both have been compared with C, an adjusted indirect comparison can sometimes estimate A versus B through the common comparator C. Network meta-analysis extends this logic to networks containing multiple interventions.

However, this is not simply arithmetic with study averages. A central requirement is transitivity: broadly, the different sets of studies should be sufficiently comparable with respect to factors that could modify the relative intervention effects.

For example, suppose A versus C trials include participants with mild disease while B versus C trials include participants with severe disease, and disease severity modifies treatment effects. The indirect A-versus-B comparison may then be misleading.

The common comparator itself should also be sufficiently similar across comparisons. If C represents intensive usual care in one set of studies and minimal usual care in another, the apparent connection in the evidence network may conceal an important clinical difference.

Different comparators are only one dimension of compatibility

Even identical comparators do not guarantee that studies should be pooled. Populations, intervention versions, outcomes, follow-up periods, designs, and risk of bias may still differ.

For example, studies may compare exactly the same treatments but assess the outcome at substantially different stages. In that situation, you also need to consider whether different outcome measurement times represent compatible effects.

Likewise, an overall treatment contrast may conceal variation across populations. If effects plausibly differ between participant groups, an average effect may obscure important subgroup differences.

How can the comparator change what an intervention effect means?

The comparator is therefore not background scenery. It is part of the effect being estimated.

Comparator mistakes that can distort an evidence synthesis

Treat the comparator as part of the intervention question

Extract comparison conditions with nearly the same care you give the intervention itself. Record their content, intensity, frequency, co-interventions, implementation, and any other features likely to affect outcomes.

Then define which comparisons are conceptually compatible before deciding how to synthesize them.

This approach also keeps your language precise. Instead of writing “the intervention improves outcomes,” specify the comparison: “the intervention improved outcomes relative to no additional support,” or “little difference was observed compared with the active alternative.” The extra few words often contain most of the information needed to interpret the result.

Finally, do not allow comparator differences to become a backdoor way of mixing distinct inferential questions. As with descriptive and causal evidence, the goal is not to force every eligible study into one analysis. It is to produce summaries whose meaning remains intact after the studies have been combined.

Before combining studies with different comparison groups, check:

Common questions about different comparison groups

An effect only makes sense relative to its comparator

How to Cite This Guide

This guide is intended to be read, shared, and used in research, teaching, and academic work. If you draw on its ideas, explanations, or other content, please acknowledge the source by citing the guide. Doing so gives appropriate credit and helps your readers locate the original resource.