Should You Broaden the Population When Direct Research Evidence Is Scarce?

Should you include studies from a broader population when direct evidence is scarce?

Your research question concerns a specific population, but only a handful of studies have examined it directly. Meanwhile, many more studies exist in people who are similar, but not quite the same.

Should you use them?

This is one of the most consequential decisions in a sparse evidence synthesis. Broadening the population may give you considerably more evidence, but it also introduces an assumption: that findings observed in one population can reasonably inform conclusions about another.

The question is therefore not simply whether the broader studies are related. It is whether the differences between their participants and your target population could plausibly change the phenomenon, association, intervention effect, baseline risk, experience, or outcome that you are trying to understand.

Broaden the population only when the transfer is defensible

Population broadening is really a question about applicability

Population is one of the core elements of a research question. In PICO-based evidence synthesis, it defines the people, participants, or population to whom the question applies. Cochrane guidance recommends defining population eligibility criteria broadly enough to encompass relevant diversity while remaining sufficiently narrow for the combined evidence to provide a meaningful answer.

When direct evidence is scarce, widening that boundary can be reasonable. But every expansion potentially changes the relationship between the evidence you have and the population you actually care about.

Start with the target population, not the available studies

Before deciding whether another population is close enough, define the target population clearly. Depending on the question, relevant characteristics might include age, condition or diagnosis, disease severity, comorbidities, socioeconomic circumstances, prior exposure, professional role, educational level, care setting, geography, or other contextual characteristics.

Not every characteristic deserves equal weight. The important characteristics are those that could plausibly affect the phenomenon or outcome under investigation.

Suppose your question concerns an intervention for older adults with a chronic condition, but most studies enrolled younger adults with the same condition. The age difference matters only insofar as it could alter factors relevant to the question, such as baseline risk, intervention response, adverse effects, adherence, or feasibility. GRADE treats differences between the population of interest and the populations actually studied as a potential source of indirectness.

Similarity is not the same as interchangeability

Two populations can appear similar while differing in a characteristic that materially affects the answer. Conversely, populations can differ in obvious demographic characteristics without those differences necessarily changing the finding of interest.

This is why population broadening should not be governed by labels alone. “Adults” and “older adults,” “university students” and “senior high school students,” or “patients with mild disease” and “patients with severe disease” tell you that populations differ. They do not, by themselves, tell you whether evidence can reasonably be transferred between them.

The substantive question is whether the difference is likely to matter for the relationship being investigated.

Ask what could modify the finding

A useful way to judge population broadening is to identify plausible effect modifiers or other characteristics that could change the answer. The relevant modifiers depend entirely on the research question.

For an intervention review, these could include baseline risk, disease severity, age, previous treatment, comorbidities, setting, or biological characteristics. For educational research, prior knowledge, developmental stage, educational level, access to technology, instructional setting, or institutional context might matter. For qualitative evidence, cultural setting, professional role, lived experience, or organizational context may shape how a phenomenon is experienced.

The point is not to prove that two populations are identical. They rarely are. Instead, determine whether there is a credible reason to expect that the differences would materially change the conclusion.

Think in terms of a gradient of directness

Population relevance is rarely binary. Evidence may range from highly direct to increasingly indirect.

GRADE similarly conceptualizes indirectness as a mismatch between the target PICO and the PICO represented by the available evidence, rather than simply asking whether studies are “relevant” in a general sense.

Do not broaden simply because more studies become available

A common temptation is to compare two searches: the narrow population gives four studies, while the broader population gives forty. Forty feels methodologically safer.

That arithmetic can be misleading. Ten times as many studies do not necessarily provide ten times as much useful evidence for the original population. If the additional studies concern participants whose characteristics materially alter applicability, the review becomes larger while the answer to the original question remains uncertain.

This is the same principle that applies whenever only a few directly relevant studies exist: study count should not determine relevance.

You do not necessarily have to choose between including and excluding everything

Population broadening does not always require placing every population into one analytical group. A broader review can sometimes include multiple populations while keeping important groups separate during synthesis.

Cochrane distinguishes between the overall review PICO, the PICO for a particular synthesis, and the PICO actually represented in included studies. This permits a review to have reasonably broad scope while still organizing syntheses around meaningful population differences.

For example, you might include studies of adolescents and adults within the review but synthesize them separately if age could plausibly modify the effect. Alternatively, broader-population evidence might be discussed as supporting or contextual evidence rather than combined directly with the target population.

What you should avoid is allowing a broad eligibility criterion to erase a population distinction that matters scientifically.

Changes made after seeing the evidence require particular care

If the population was defined prospectively in a protocol, broadening it after discovering few eligible studies is a methodological amendment. That does not make the change automatically inappropriate, but it does make transparency important.

Cochrane guidance emphasizes prespecification of participant eligibility criteria and documentation of changes made during the review.

Explain what changed, why the original population definition proved unnecessarily restrictive or why indirect evidence became necessary, and how the additional population will be handled analytically. This allows readers to distinguish a reasoned methodological adjustment from a post-hoc attempt to obtain more studies.

What if evidence exists in adults but your question concerns adolescents?

Notice what population broadening accomplished here. It provided additional information, but it did not make the evidence directly adolescent simply by placing it inside the same review.

Population broadening can look simpler than it is

Ask whether population differences could change the answer

The most useful decision rule is not “How similar are these populations?” but “Could the ways in which they differ materially change the finding I want to apply?”

Population is also only one route through which researchers may broaden sparse evidence. Depending on the question, you may instead need to consider whether it is appropriate to broaden the outcomes being considered or include evidence from related conditions, interventions, or contexts. Each decision creates its own assumptions and should be justified separately.

Before including a broader population

Questions about using evidence from broader populations

Broaden populations based on transferability, not study count

How to Cite This Guide

This guide is intended to be read, shared, and used in research, teaching, and academic work. If you draw on its ideas, explanations, or other content, please acknowledge the source by citing the guide. Doing so gives appropriate credit and helps your readers locate the original resource.