Should You Include a Study With the Right Outcome but Different Population?

What if the outcome is perfect but the participants are not?

You find a study measuring exactly the outcome your review needs. The intervention or exposure may also fit beautifully. There is only one problem: the participants are not quite the population specified in your research question.

Perhaps your review concerns undergraduate students, but the study combines undergraduate and postgraduate students. Your review concerns older adults, but the sample begins at age 50. Or your target population is patients with a particular diagnosis, while the paper includes a broader clinical group.

These cases can be frustrating because the study may look highly informative. Yet a matching outcome cannot by itself compensate for a population that falls outside predefined eligibility criteria. The real question is whether the population is genuinely ineligible, partially eligible, or simply different in a way your criteria already permit.

Start with the population criteria, not the attractiveness of the outcome

Population is often a defining boundary of the review question

The population tells you whom your review is about

Population criteria identify the people, participants, cases, settings, or other units to which the review question applies. In an intervention review, this is the P in PICO: population, intervention, comparator, and outcome.

Cochrane guidance recommends defining eligibility criteria prospectively and notes that population characteristics commonly form part of the basis for deciding which studies are eligible. Population criteria may involve a condition or diagnosis, age, demographic characteristics, setting, disease severity, previous treatment, or other characteristics relevant to the review question.

This means the right outcome is only one part of the eligibility problem. If your review concerns mathematics achievement among primary-school pupils, a study reporting precisely the same achievement outcome among university students is not automatically eligible.

Different does not necessarily mean ineligible

Do not equate any population difference with exclusion. The relevant comparison is between the study population and your operational eligibility criteria, not between the study and an imagined perfect sample.

Suppose your review includes adults aged 18 years and older. One study has a mean participant age of 23 and another a mean age of 67. Those populations differ substantially, but both may satisfy the criterion.

Likewise, if your review includes higher-education students, both undergraduate and postgraduate populations may qualify. If it explicitly includes undergraduates only, however, the distinction becomes consequential.

The first situation may create heterogeneity or applicability questions later. The second is a study-selection issue.

A mixed population requires a more careful decision

Many studies contain both eligible and ineligible participants. A paper may combine adolescents and adults, undergraduate and postgraduate students, several diagnostic groups, or participants from multiple settings.

Do not immediately classify the entire study according to whichever group is most visible in the abstract. Determine whether the eligible subgroup can be identified and whether its data are reported separately.

Cochrane guidance recognizes that reviews sometimes encounter studies containing only a subset of relevant participants. Review authors should specify in advance how such studies will be handled, including any rules concerning the proportion of participants who must meet the population criteria or whether separate data for the eligible subgroup are required.

This is fundamentally different from quietly changing the population definition after seeing a useful paper.

Separately extractable subgroup data can matter

Suppose a study contains 300 university students, of whom 220 are undergraduates and 80 are postgraduates. Your review includes undergraduates only.

If results for the 220 undergraduates are reported separately and the subgroup otherwise satisfies your criteria, those data may be usable if your protocol permits this approach. If all 300 participants are pooled and no separate undergraduate results are available, the decision depends on the rules you established for mixed populations.

Possible approaches include requiring all participants to be eligible, accepting studies when a specified proportion of the sample qualifies, or including only separately extractable eligible subgroups. There is no defensible universal percentage that can simply be borrowed for every review. The threshold should arise from the review question and methodology rather than convenience.

The right outcome cannot repair a wrong population

Researchers can become attached to a paper because it measures the outcome unusually well. That creates a subtle temptation to relax another criterion.

Suppose your review asks whether an intervention reduces medication errors among registered nurses, but an otherwise ideal study includes only medical students. The outcome may be identical, and the intervention may be similar, yet the participants represent a substantively different population.

If registered-nurse status defines eligibility, the study remains outside the evidence base. It might still provide useful background evidence, support discussion of mechanisms, or motivate future research. Inclusion in the formal synthesis, however, should follow the eligibility criteria.

This is the mirror image of finding the correct population but an unexpected outcome. The two situations should not automatically be treated identically because different review components can play different roles in study eligibility.

Population eligibility is different from applicability

An important distinction emerges once a study is eligible. Two eligible populations can still differ enough that the findings may apply differently to the population of primary interest.

Cochrane's guidance on GRADE describes indirectness as a concern when the evidence differs from the review question in characteristics such as population, intervention, comparator, or outcome. Thus, a population can sometimes fall within a review's deliberately broad inclusion criteria while still raising questions about how directly its evidence answers a narrower decision problem.

Eligibility asks whether the study enters the review. Applicability or indirectness asks how directly its evidence informs the question being interpreted. Collapsing these into one decision can lead either to unnecessary exclusions or to overconfident generalization.

Geography alone does not necessarily define a different population

A study conducted in another country should not automatically be treated as a study of the wrong population. Country, healthcare system, culture, socioeconomic conditions, educational system, or other contextual characteristics may matter, but their importance depends on the review question.

If geographic location is an explicit and justified eligibility criterion, apply it. If not, the study may remain eligible while contextual differences are considered during synthesis and interpretation. The broader question of whether evidence from another country or population should be excluded therefore requires more than noticing that the study was conducted elsewhere.

Do not redefine the population one attractive paper at a time

If repeated screening reveals that your population criteria are poorly specified, revising them may sometimes be methodologically defensible. What matters is how the change is made.

A revision should arise from a reason applicable to the review as a whole, not from a desire to retain one study. Apply the revised rule to all records that could be affected, document the change, and consider whether previously screened records need reassessment.

That is different from selectively relaxing a criterion because a particular study has an appealing outcome. If the underlying issue is a genuine protocol amendment, the principles governing changes to inclusion criteria after screening has started become relevant.

How to handle a mixed population with the exact outcome you need

Common mistakes when the outcome fits but the population does not

Determine whether the population is different, mixed, or genuinely ineligible

When a study has your preferred outcome, temporarily set that attraction aside and apply the population criterion exactly as you would if the outcome were less exciting.

If you ultimately exclude the study, record the substantive reason as a population mismatch rather than something vague such as "not relevant." Clear reasons make later auditing easier and support consistent documentation of excluded studies.

Before including a study with a different population, check this

Questions about studies with the right outcome but different population

The right outcome does not automatically make the population right

How to Cite This Guide

This guide is intended to be read, shared, and used in research, teaching, and academic work. If you draw on its ideas, explanations, or other content, please acknowledge the source by citing the guide. Doing so gives appropriate credit and helps your readers locate the original resource.