When Is Evidence From Another Population Directly Relevant to Yours?

How similar does another population need to be before its evidence applies to yours?

The ideal study rarely matches your target population perfectly. Perhaps the available evidence comes from another country, a somewhat younger population, specialist hospitals rather than community clinics, or participants with different baseline characteristics.

Does that make the evidence indirect? Sometimes. But requiring an exact match would make remarkably little research usable outside the original sample.

The practical question is not whether the populations differ. They almost always do. It is whether those differences are likely to change the effect or finding you care about.

Direct relevance depends on effect-relevant similarity, not identical populations

No research population perfectly matches the population where evidence will be used

Generalizing research always involves some movement from the people who were studied to people who were not. Cochrane notes that no individual can be entirely matched to a research population and that applying evidence therefore requires judgments about how closely the evidence matches the question at hand.

That observation changes how population relevance should be assessed. The standard cannot be “Are these populations identical?” If it were, virtually every application of research would fail.

Instead ask: “Are they different in ways likely to matter for this particular finding?”

Define the target population before judging relevance

You cannot determine whether evidence is direct without specifying what it is supposed to be direct for.

A review might concern all adults with a condition, adults receiving primary care, adolescents in public schools, first-year university students, rural households, or another explicitly defined population. Those targets create different standards of relevance.

GRADE formalizes this through the target PICO: the population, intervention, comparator, and outcome defining the question. Indirectness arises when important mismatches exist between that target and the studies providing the evidence.

So begin with the target. Otherwise, “applicable” has no stable reference point.

Not every population difference is an effect modifier

Suppose the average age differs by five years between the study population and yours. That difference is real, but it matters only if age is likely to change the finding sufficiently to affect the inference.

The same reasoning applies to sex, gender, ethnicity, socioeconomic position, geography, disease severity, comorbidity, education, prior exposure, and many other characteristics.

Core GRADE guidance asks whether substantial differences between the target and study populations create a likelihood that the magnitude of effect will differ substantially. Population mismatch alone is therefore not sufficient.

Look for characteristics that could modify the effect

The most useful comparison focuses on plausible effect modifiers rather than producing a long inventory of demographic differences.

Depending on the question, these might include baseline risk, age, disease severity, comorbidity, prior treatment, developmental stage, exposure intensity, socioeconomic conditions, institutional environment, or access to supporting resources.

The same characteristic can be crucial in one question and irrelevant in another. Age might strongly influence the effect of a developmental intervention while contributing little to another phenomenon within the observed range.

This is why the more focused questions of how evidence behaves across age groups and whether effects differ across sex or gender groups require substantive reasoning rather than automatic demographic stratification.

Distinguish relative effects from absolute effects

Populations can experience different absolute consequences even when relative effects transfer reasonably well.

Suppose an intervention reduces the relative risk of an outcome by approximately the same proportion across populations, but the outcome is much more common in your target population. The absolute benefit may then be substantially larger.

Conversely, a population with very low baseline risk may experience a small absolute benefit even when the relative effect is similar.

Population applicability therefore cannot always be summarized as “the effect transfers” or “the effect does not transfer.” The relevant effect measure matters.

Population is only one part of directness

An apparently similar population does not guarantee directly applicable evidence if other components of the research question differ.

The intervention may be delivered at a different intensity. The comparator may represent a substantially different standard of care. Outcomes may be measured differently. Follow-up may be too short for the target decision.

Core GRADE treats mismatches in population, intervention, comparator, and outcome as potential sources of indirectness.

This matters because researchers sometimes focus intensely on demographic similarity while overlooking a much more consequential difference in intervention delivery or comparator conditions.

Setting can change whether otherwise similar populations are comparable

Two groups of participants may look similar demographically while receiving an intervention within very different systems.

A program evaluated in a specialist center might depend on personnel, technology, supervision, or referral services unavailable in your setting. Cochrane specifically identifies such contextual differences as potential applicability concerns and cautions against assuming that programs successfully transfer unchanged between contexts.

When resources are central to implementation, the question of how evidence transfers across different resource settings may matter more than superficial demographic similarity.

Evidence from another country is not automatically indirect

National borders are weak proxies for causal relevance. Populations in different countries can be highly similar on the characteristics that matter to a particular finding, while populations within one country can differ substantially.

Cultural, linguistic, socioeconomic, institutional, rural or urban, and service-system characteristics can affect applicability in some questions. Cochrane specifically identifies several of these contextual dimensions when discussing indirectness and applicability.

The appropriate approach is therefore to identify the contextual characteristic rather than use country as a substitute for it.

Representation and applicability are related but different

If your target population was poorly represented in the evidence, concern about applicability may increase. But underrepresentation does not automatically prove that the effect differs.

Conversely, including some members of the target group does not establish that the evidence is adequate for them. A trial containing 5% older adults may technically include older participants while providing very limited information about effects in that population.

Ask both questions: Was the target population represented, and is there reason to expect its relevant characteristics to modify the finding?

Consistency across different populations can be informative

Suppose comparable effects repeatedly appear among populations that differ substantially on characteristics initially suspected to matter. That pattern can reduce concern that the finding is confined to one narrow group.

It does not establish universality. However, consistent effects across genuinely different populations and settings can provide useful evidence about robustness.

Conversely, systematic differences may indicate that population characteristics matter and deserve preservation rather than averaging. Applicability is therefore an empirical question where evidence exists, not merely a judgment based on resemblance.

When foreign evidence may be more relevant than local evidence

This does not mean Study A automatically provides the better evidence. Cultural, curricular, linguistic, or other differences might still matter. The point is that relevance has to be argued from effect-relevant characteristics rather than inferred from the national label.

Common mistakes when deciding whether evidence applies

Compare mechanisms and effect modifiers, not demographic profiles alone

When deciding whether another population's evidence is directly relevant, define your target population first. Then identify characteristics that could realistically change the finding and compare those characteristics with the evidence base.

There will not always be a sharp threshold between direct and indirect evidence. GRADE treats indirectness as a judgment about how likely mismatches are to produce substantially different effects.

When the mismatch becomes too consequential to defend, the appropriate conclusion may be that the evidence should not be assumed to generalize to the target population.

Before applying evidence from another population, check:

Questions about applying evidence to another population

Relevant populations do not need to be identical populations

How to Cite This Guide

This guide is intended to be read, shared, and used in research, teaching, and academic work. If you draw on its ideas, explanations, or other content, please acknowledge the source by citing the guide. Doing so gives appropriate credit and helps your readers locate the original resource.